nerves_system_br packages built with v0.17.3 and later fail to untar (missing directories?) #631
Labels
Comments
|
Does it matter which Hex version you use when fetching the package, do they all fail with the same error? |
|
Hmm. I don't know I tried it with two hex versions - v0.18.2 and one other. Both failed, but I didn't take notes. I got fixated on |
ericmj
added a commit
that referenced
this issue
Nov 28, 2018
Symlinks should be added to the end of the tarball to avoid extracting through symlinks. Fixes #631.
ericmj
added a commit
that referenced
this issue
Nov 28, 2018
Symlinks should be added to the end of the tarball to avoid extracting through symlinks. Fixes #631.
ericmj
added a commit
that referenced
this issue
Nov 28, 2018
Symlinks should be added to the end of the tarball to avoid extracting through symlinks. Fixes #631.
ericmj
added a commit
that referenced
this issue
Nov 28, 2018
Symlinks should be added to the end of the tarball to avoid extracting through symlinks. Fixes #631.
ericmj
added a commit
that referenced
this issue
Nov 28, 2018
Symlinks should be added to the end of the tarball to avoid extracting through symlinks. Fixes #631.
fhunleth
added a commit
to fhunleth/hex
that referenced
this issue
Nov 28, 2018
Path.wildcard("**") follows symlinks when collecting all of the files
under a path. This can lead to the following undesirable behavior:
1. If a file under a symlinked directory is added before the actual
directory, then tarball extraction will fail on that file. (The
directory will not have been created yet.
2. If a symlink points to a directory outside of the main project, then
unintended files may be pulled in.
Fixes hexpm#631.
fhunleth
added a commit
to fhunleth/hex
that referenced
this issue
Nov 28, 2018
Path.wildcard("**") follows symlinks when collecting all of the files
under a path. This can lead to files in the tarball containing paths
with the symlink in them. If the directory corresponding to the symlink
hasn't been created, then the extraction will fail.
As an example, the "create with files" unit test now contains a symlink
in a directory. Without the fix, it can fail like this:
```
1) test create with files (Mix.Tasks.Hex.BuildTest)
test/mix/tasks/hex.build_test.exs:42
** (MatchError) no match of right hand side value: {:error, :eexist}
code: in_tmp(fn ->
stacktrace:
test/mix/tasks/hex.build_test.exs:13: Mix.Tasks.Hex.BuildTest.extract/2
test/mix/tasks/hex.build_test.exs:69: anonymous fn/0 in Mix.Tasks.Hex.BuildTest."test create with files"/1
(elixir) lib/file.ex:1443: File.cd!/2
test/mix/tasks/hex.build_test.exs:45: (test)
```
Untaring the `contents.tar.gz` shows the problem.
```sh
$ tar tfz contents.tar.gz
myfile.txt
executable.sh
dir/.dotfile
dir/a_link_to_dir2
dir/a_link_to_dir2/test.txt
dir/dir2/test.txt
empty_dir/
link_dir
```
`dir/a_link_to_dir2` is created as a symlink to `dir/dir2`. The
`test.txt` file is then extracted to it. This fails since `dir2` hasn't
been created yet so `a_link_to_dir2` is dangling. It's also not
desirable that `test.txt` was included twice.
After the fix, the `contents.tar.gz` looks like this:
```sh
$ tar tfz contents.tar.gz
myfile.txt
executable.sh
dir/a_link_to_dir2
dir/dir2/test.txt
dir/.dotfile
empty_dir/
link_dir
```
Fixes hexpm#631.
fhunleth
added a commit
to fhunleth/hex
that referenced
this issue
Nov 28, 2018
Path.wildcard("**") follows symlinks when collecting all of the files
under a path. This can lead to files in the tarball containing paths
with the symlink in them. If the directory corresponding to the symlink
hasn't been created, then the extraction will fail.
As an example, the "create with files" unit test now contains a symlink
in a directory. Without the fix, it fails like this:
```
1) test create with files (Mix.Tasks.Hex.BuildTest)
test/mix/tasks/hex.build_test.exs:42
** (MatchError) no match of right hand side value: {:error, :eexist}
code: in_tmp(fn ->
stacktrace:
test/mix/tasks/hex.build_test.exs:13: Mix.Tasks.Hex.BuildTest.extract/2
test/mix/tasks/hex.build_test.exs:69: anonymous fn/0 in Mix.Tasks.Hex.BuildTest."test create with files"/1
(elixir) lib/file.ex:1443: File.cd!/2
test/mix/tasks/hex.build_test.exs:45: (test)
```
Untaring the `contents.tar.gz` shows the problem.
```sh
$ tar tfz contents.tar.gz
myfile.txt
executable.sh
dir/.dotfile
dir/a_link_to_dir2
dir/a_link_to_dir2/test.txt
dir/dir2/test.txt
empty_dir/
link_dir
```
`dir/a_link_to_dir2` is created as a symlink to `dir/dir2`. The
`test.txt` file is then extracted to it. This fails since `dir2` hasn't
been created yet so `a_link_to_dir2` is dangling. It's also not
desirable that `test.txt` was included twice.
This test seems like it is dependent on the order that the OS lists
files in a directory. My tests are on Linux, but OSX orders files
differently. The symlink starts with an "a" to try to force it to list
first.
After the fix in this commit, the `contents.tar.gz` looks like this:
```sh
$ tar tfz contents.tar.gz
myfile.txt
executable.sh
dir/a_link_to_dir2
dir/dir2/test.txt
dir/.dotfile
empty_dir/
link_dir
```
As you can see, `test.txt` is listed once and its parent directory is
the real directory, `dir2`, and not the symlink.
Fixes hexpm#631.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Creating a new nerves_system_br package causes "inner tarball errors" when upgrading projects:
To reproduce, clone the
nerves_system_brpackage and run:This works with hex v0.17.1. hex v0.17.2 fails to build a tarball and v0.17.3 and later build tarballs that don't extract due to the above errors. It appears that some directories are not being added to the tarball. This is strange, but maybe it's due to the fact that the only file in the directories that fail begins with a
.. Could it be that there's some code that checks whether a directory is empty for whether to include it or not and that code doesn't check for dot files?The text was updated successfully, but these errors were encountered: