os: add support for long path names on unix RemoveAll #28494

ostenbom · 2018-10-30T23:00:30Z

On unix systems, long enough path names will fail when performing syscalls
like Lstat. The current RemoveAll uses several of these syscalls, and so
will fail for long paths. This can be risky, as it can let users "hide"
files from the system or otherwise make long enough paths for programs
to fail. By using Unlinkat and Openat syscalls instead, RemoveAll is
safer on unix systems. Initially implemented for linux, darwin, dragonfly,
netbsd and openbsd. Not yet implemented on freebsd due to fstatat 64-bit
inode compatibility issues.

Fixes #27029

Co-authored-by: Giuseppe Capizzi gcapizzi@pivotal.io
Co-authored-by: Julia Nedialkova yulia.nedyalkova@sap.com

googlebot · 2018-10-30T23:00:32Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

ostenbom · 2018-10-30T23:01:05Z

Consent verified earlier in #27871

googlebot · 2018-10-30T23:03:53Z

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

gopherbot · 2018-10-30T23:05:02Z

This PR (HEAD: 240cc3d) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/146020 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

gopherbot · 2018-10-30T23:08:01Z

Message from Gerrit User 5340:

Patch Set 1: Run-TryBot+1 Code-Review+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:08:26Z

Message from Gerrit User 5976:

Patch Set 1:

TryBots beginning. Status page: https://farmer.golang.org/try?commit=b8300245

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:14:01Z

Message from Gerrit User 5976:

Patch Set 1:

Build is still in progress...
This change failed on freebsd-amd64-11_1:
See https://storage.googleapis.com/go-build-log/b8300245/freebsd-amd64-11_1_6b2c807d.log

Consult https://build.golang.org/ to see whether it's a new failure. Other builds still in progress; subsequent failure notices suppressed until final report.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

googlebot · 2018-10-30T23:16:02Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

gopherbot · 2018-10-30T23:16:18Z

This PR (HEAD: f1fe8cc) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/146020 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

googlebot · 2018-10-30T23:36:59Z

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

gopherbot · 2018-10-30T23:37:18Z

Message from Gerrit User 5976:

Patch Set 1: TryBot-Result-1

1 of 19 TryBots failed:
Failed on freebsd-amd64-11_1: https://storage.googleapis.com/go-build-log/b8300245/freebsd-amd64-11_1_6b2c807d.log

Consult https://build.golang.org/ to see whether they are new failures.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:19Z

Message from Gerrit User 29433:

Patch Set 2:

Forgot to skip long path test for freebsd, this patch should be good.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:19Z

Message from Gerrit User 19560:

Patch Set 2: Run-TryBot+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:20Z

Message from Gerrit User 5976:

Patch Set 2:

TryBots beginning. Status page: https://farmer.golang.org/try?commit=897dc2b8

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:20Z

Message from Gerrit User 5976:

Patch Set 2: TryBot-Result+1

TryBots are happy.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:21Z

Message from Gerrit User 29433:

Patch Set 2:

(1 comment)

Remove at_sysnum_freebsd.go?

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:37:28Z

This PR (HEAD: f267de5) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/146020 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

gopherbot · 2018-10-30T23:38:17Z

Message from Gerrit User 12446:

Uploaded patch set 3: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:39:53Z

Message from Gerrit User 5206:

Patch Set 3:

(1 comment)

According to https://build.golang.org, the earlier patch failed on Darwin. We should use the old non-at code on Darwin for now, unless you have reason to believe that the problem is fixed.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:46:46Z

Message from Gerrit User 29433:

Patch Set 3:

Patch Set 3:

(1 comment)

According to https://build.golang.org, the earlier patch failed on Darwin. We should use the old non-at code on Darwin for now, unless you have reason to believe that the problem is fixed.

I have changed the Darwin syscall number to be the fstatat64 version, which works on my Darwin machine and in the x/says package from the looks of it, so I think we are ok

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

On unix systems, long enough path names will fail when performing syscalls like `Lstat`. The current RemoveAll uses several of these syscalls, and so will fail for long paths. This can be risky, as it can let users "hide" files from the system or otherwise make long enough paths for programs to fail. By using `Unlinkat` and `Openat` syscalls instead, RemoveAll is safer on unix systems. Initially implemented for linux, darwin and openbsd. Not yet implemented on freebsd due to fstatat 64-bit inode compatibility issues. Fixes golang#27029 Co-authored-by: Giuseppe Capizzi <gcapizzi@pivotal.io> Co-authored-by: Julia Nedialkova <yulia.nedyalkova@sap.com>

gopherbot · 2018-10-30T23:48:10Z

This PR (HEAD: 9235489) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/146020 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

googlebot · 2018-10-30T23:48:59Z

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

gopherbot · 2018-10-30T23:49:22Z

Message from Gerrit User 29433:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:49:22Z

Message from Gerrit User 12446:

Uploaded patch set 4: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:53:23Z

Message from Gerrit User 5206:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-30T23:56:14Z

Message from Gerrit User 29433:

Patch Set 4:

Patch Set 4:

(1 comment)

The commit message has changed: https://github.com/golang/go/pull/28494/commits
Gerrit just doesn't seem to pick it up.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:16:57Z

Message from Gerrit User 5206:

Patch Set 5: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:18:02Z

Message from Gerrit User 5206:

Patch Set 5: Run-TryBot+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:18:02Z

Message from Gerrit User 5976:

Patch Set 5:

TryBots beginning. Status page: https://farmer.golang.org/try?commit=77363085

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:18:03Z

Message from Gerrit User 12446:

Uploaded patch set 6: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:25:57Z

Message from Gerrit User 5976:

Patch Set 5: TryBot-Result+1

TryBots are happy.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T00:29:34Z

Message from Gerrit User 5206:

Patch Set 6:

+andybons help: how do we fix the commit message?

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T05:29:38Z

Message from Gerrit User 5065:

Patch Set 6:

Patch Set 6:

+andybons help: how do we fix the commit message?

See the bottom of https://github.com/golang/go/wiki/CommitMessage#github-pull-requests

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T08:55:23Z

Message from Gerrit User 12446:

Uploaded patch set 7: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T09:12:31Z

Message from Gerrit User 19560:

Patch Set 7:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T10:54:45Z

Message from Gerrit User 12446:

Uploaded patch set 8: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T10:56:36Z

Message from Gerrit User 12446:

Uploaded patch set 9: Commit message was updated.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T10:57:10Z

Message from Gerrit User 29433:

Patch Set 9:

(1 comment)

Patch Set 7:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T10:58:32Z

Message from Gerrit User 19560:

Patch Set 9: Run-TryBot+1

Thanks

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T10:59:07Z

Message from Gerrit User 5976:

Patch Set 9:

TryBots beginning. Status page: https://farmer.golang.org/try?commit=aac8f9ab

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T11:07:20Z

Message from Gerrit User 5976:

Patch Set 9: TryBot-Result+1

TryBots are happy.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2018-10-31T12:23:41Z

Message from Gerrit User 19560:

Patch Set 9: Code-Review+1

LGTM. Leaving to Ian or Brad for final review.

Please don’t reply on this GitHub thread. Visit golang.org/cl/146020.
After addressing review feedback, remember to publish your drafts!

On unix systems, long enough path names will fail when performing syscalls like `Lstat`. The current RemoveAll uses several of these syscalls, and so will fail for long paths. This can be risky, as it can let users "hide" files from the system or otherwise make long enough paths for programs to fail. By using `Unlinkat` and `Openat` syscalls instead, RemoveAll is safer on unix systems. Initially implemented for linux, darwin, dragonfly, netbsd and openbsd. Not yet implemented on freebsd due to fstatat 64-bit inode compatibility issues. Fixes #27029 Co-authored-by: Giuseppe Capizzi <gcapizzi@pivotal.io> Co-authored-by: Julia Nedialkova <yulia.nedyalkova@sap.com> Change-Id: I978a6a4986878fe076d3c7af86e7927675624a96 GitHub-Last-Rev: 9235489 GitHub-Pull-Request: #28494 Reviewed-on: https://go-review.googlesource.com/c/146020 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot · 2018-10-31T13:56:45Z

This PR is being closed because golang.org/cl/146020 has been merged.

googlebot added the cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. label Oct 30, 2018

ianlancetaylor added cla: yes Used by googlebot to label PRs as having a valid CLA. The text of this label should not change. and removed cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. labels Oct 30, 2018

ostenbom force-pushed the os-removeall-long-path-names branch from 240cc3d to f1fe8cc Compare October 30, 2018 23:15

googlebot added cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. and removed cla: yes Used by googlebot to label PRs as having a valid CLA. The text of this label should not change. labels Oct 30, 2018

ostenbom force-pushed the os-removeall-long-path-names branch from f1fe8cc to f267de5 Compare October 30, 2018 23:35

ianlancetaylor added cla: yes Used by googlebot to label PRs as having a valid CLA. The text of this label should not change. and removed cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. labels Oct 30, 2018

googlebot added cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. and removed cla: yes Used by googlebot to label PRs as having a valid CLA. The text of this label should not change. labels Oct 30, 2018

ianlancetaylor added cla: yes Used by googlebot to label PRs as having a valid CLA. The text of this label should not change. and removed cla: no Used by googlebot to label PRs as having an invalid CLA. The text of this label should not change. labels Oct 30, 2018

ostenbom changed the title ~~os: Add support for long path names on unix RemoveAll~~ os: add support for long path names on unix RemoveAll Oct 31, 2018

gopherbot closed this Oct 31, 2018

os: add support for long path names on unix RemoveAll #28494

os: add support for long path names on unix RemoveAll #28494

Conversation

ostenbom commented Oct 30, 2018 • edited Loading

googlebot commented Oct 30, 2018

ostenbom commented Oct 30, 2018

googlebot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

googlebot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

googlebot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

googlebot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

gopherbot commented Oct 31, 2018

ostenbom commented Oct 30, 2018 •

edited

Loading