Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

InodeDelta comparison of FullType doesn't line up #146

Open
vbatts opened this issue Dec 13, 2017 · 5 comments
Open

InodeDelta comparison of FullType doesn't line up #146

vbatts opened this issue Dec 13, 2017 · 5 comments

Comments

@vbatts
Copy link
Owner

vbatts commented Dec 13, 2017

#145 (comment)

gomtree -f <(casync mtree .) is a good example of this.
The output of casync parses just fine, but does not validate just fine.

cc @cyphar @poettering
@vbatts
Copy link
Owner Author

vbatts commented Dec 13, 2017

example output

"cmd/gomtree/hierarchy_test.go": missing path
"README.md": unexpected path
"keywords_linux_test.go": unexpected path
".git/refs/tags/.travis.yml": missing path
"cmd/gomtree/mtree_test.go": missing path
"testdata/traversal/dir2/dir3/actualdir2/update.go": missing path
"glide.lock": unexpected path
"glide.yaml": unexpected path
"compare.go": unexpected path
".git/refs/tags/.vscode": missing path
"testdata/traversal/dir2/dir3/actualdir2/updatefuncs.go": missing path
"tar.go": unexpected path

@vbatts
Copy link
Owner Author

vbatts commented Dec 13, 2017 •
edited

https://github.com/vbatts/go-mtree/blob/master/testdata/source.casync-mtree is an example of the mtree output

@vbatts vbatts mentioned this issue Dec 13, 2017
Merged
@cyphar
Copy link
Contributor

cyphar commented Sep 14, 2018

After looking at this quite a bit, I've become convinced it's actually a casync bug -- they aren't generating mtree-compatible output. I've opened systemd/casync#167 with proposals for fixing it and examples of what happens. The simple trick of adding .. after each type=dir causes validation to work with gomtree, but FreeBSD mtree disallows / in node names and so we need to have a more complete solution.

@cyphar cyphar mentioned this issue Sep 14, 2018
Closed
@cyphar cyphar mentioned this issue Jun 3, 2023
Open
@cyphar
Copy link
Contributor

cyphar commented Jun 4, 2023

Hmmm. I've looked at this again. While I was wrong to say the format is not valid, casync prints every entry as though it's a FullType but it will happily output directory entries that do not have / (which makes them relative and thus affects the current directory). However I think there's a separate issue in go-mtree (FullTypes are treated as being able to change the current directory, which doesn't match the "spec"). I also have a feeling we don't handle the spec-required behaviour of allowing more than one FullType to reference the same path...

cyphar added a commit to cyphar/casync that referenced this issue Jun 4, 2023
@cyphar
mtree: append "/" to all directory entries
44f2234 
casync attempts to output all entries as a Full entry but for top-level
directories, there is no "/" in the escaped pathname, resulting in the
entry being treated as a Relative entry. As per the specification[1],
this results in any subsequent Relative entries (such as top-level
files) being treated as children of the directory which causes manifests
produced by casync to fail.

The simplest solution is to force all directories to be treated as Full
entries by adding a dummy "/" to the end of the pathname. It's not
necessary to do anything for top-level files because they do not affect
the "current directory" during parsing.

[1]: https://man.netbsd.org/mtree.5

Ref: vbatts/go-mtree#146
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
@cyphar cyphar mentioned this issue Jun 4, 2023
Open
cyphar added a commit to cyphar/casync that referenced this issue Jun 4, 2023
@cyphar
mtree: append "/" to all directory entry pathnames
3fa12c2 
casync attempts to output all entries as a Full entry but for top-level
directories, there is no "/" in the escaped pathname, resulting in the
entry being treated as a Relative entry. As per the specification[1],
this results in any subsequent Relative entries (such as top-level
files) being treated as children of the directory which causes manifests
produced by casync to fail.

The simplest solution is to force all directories to be treated as Full
entries by adding a dummy "/" to the end of the pathname. It's not
necessary to do anything for top-level files because they do not affect
the "current directory" during parsing.

[1]: https://man.netbsd.org/mtree.5

Ref: vbatts/go-mtree#146
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
@cyphar cyphar mentioned this issue Jun 4, 2023
Merged
@cyphar
Copy link
Contributor

cyphar commented Jun 4, 2023

systemd/casync#267 should fix the casync side, and #188 fixes the core issues with FullType though I still am not sure what we should do in the multiple-entries-for-the-same-path case. At the moment we just use the attributes of the last one but I'm not sure if that's the behaviour BSD expects (the "spec" doesn't tell us the correct behaviour, but I suspect our current behaviour is correct).

cyphar added a commit to cyphar/casync that referenced this issue Jun 5, 2023
@cyphar
mtree: prepend "./" to all entries other than "."
9a71693 
casync attempts to output all entries as a Full entry but for top-level
entries, there is no "/" in the escaped pathname, resulting in the entry
being treated as a Relative entry.

As per the specification[1], this results in any subsequent Relative
entries (such as top-level files) being treated as children of the
directory which causes manifests produced by casync to fail.

The simplest solution (and the one that "nmtree -C" does) is to prefix
every path with "./". However, nmtree does not like "./." (internally it
requires every entry referenced in an Full entry's path to already have
been referenced in the spec) and so we have to special-case the "."
path.

[1]: https://man.netbsd.org/mtree.5

Ref: vbatts/go-mtree#146
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vbatts @cyphar