New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Siegfried blocks on unresolved Alias (Mac OS X) #107

Closed
hvanstappen opened this Issue Sep 20, 2017 · 7 comments

Comments

Projects
None yet
3 participants
@hvanstappen

hvanstappen commented Sep 20, 2017

Hi,

I have to identify the entire contents of an internal HDD (HFS extended), but Siegfried seems to block every time it meets aliases (shortcut) that can't be resolved:
"[ERROR] open /Volumes/CK6_files/Users/christiankieckens/Library/Application Support/SyncServices/Local/clientdata/633a1ba25cb8241bbde44acb603ee1e822cde772/004100420053004c00610073007400530079006e00630044006100740065: no such file or directory"

It seems to be related with the fact that these aliases are self-referring. What puzzles me more is that SF doesn't block on the same file in every run: it may accept the first, but then blocks with the next self-referring alias.

Is there a way to prevent SF from trying to resolve aliases?

thanks,

Henk

@richardlehane richardlehane self-assigned this Sep 20, 2017

@richardlehane

This comment has been minimized.

Show comment
Hide comment
@richardlehane

richardlehane Sep 20, 2017

Owner

Thanks for this report Henk. I will take a look soon
Cheers
Richard
ps very sorry to mispell your name in original comment

Owner

richardlehane commented Sep 20, 2017

Thanks for this report Henk. I will take a look soon
Cheers
Richard
ps very sorry to mispell your name in original comment

@richardlehane

This comment has been minimized.

Show comment
Hide comment
@richardlehane

richardlehane Sep 21, 2017

Owner

Hi Henk
I've had a look at this today. Have managed to create some synthetic tests on my windows laptop using symlinked files and directories. Symlinked files work on windows but attempting to scan a symlinked directory in sf currently causes it to panic.

A proposed fix for your issue, but also the general issues that arise in scanning non-regular files (symlinks, sockets, named pipes etc.), is to change the file walk function so that only regular files are scanned. Golang has an IsRegular function (https://golang.org/pkg/os/#FileMode.IsRegular) that I could easily drop in.

When non-regular files are encountered I could include a generic error message in results so users are aware that those files have been skipped.

Given that this changes current behaviour (so for example symlinked files are currently scanned successfully by sf) I'd be interested in any feedback on this fix before it is implemented.

cheers
Richard

Owner

richardlehane commented Sep 21, 2017

Hi Henk
I've had a look at this today. Have managed to create some synthetic tests on my windows laptop using symlinked files and directories. Symlinked files work on windows but attempting to scan a symlinked directory in sf currently causes it to panic.

A proposed fix for your issue, but also the general issues that arise in scanning non-regular files (symlinks, sockets, named pipes etc.), is to change the file walk function so that only regular files are scanned. Golang has an IsRegular function (https://golang.org/pkg/os/#FileMode.IsRegular) that I could easily drop in.

When non-regular files are encountered I could include a generic error message in results so users are aware that those files have been skipped.

Given that this changes current behaviour (so for example symlinked files are currently scanned successfully by sf) I'd be interested in any feedback on this fix before it is implemented.

cheers
Richard

@hvanstappen

This comment has been minimized.

Show comment
Hide comment
@hvanstappen

hvanstappen Sep 21, 2017

hvanstappen commented Sep 21, 2017

@timothyryanwalsh

This comment has been minimized.

Show comment
Hide comment
@timothyryanwalsh

timothyryanwalsh Sep 21, 2017

Hi Richard and Henk,

I agree with Henk that having an option in Siegfried to scan only regular files makes sense. I would go so far as to say that for my use cases, it might even make sense to make this the default behavior and add to add a flag for scanning all (including non-regular) files. That is more or less what we've been doing in-house at CCA when making extent statements from DFXML manifests.

Looking forward to hearing others' opinions and use cases.

Apologies for the many edits: that's what I get for writing this before I've finished my morning coffee :)

Cheers,
Tim

timothyryanwalsh commented Sep 21, 2017

Hi Richard and Henk,

I agree with Henk that having an option in Siegfried to scan only regular files makes sense. I would go so far as to say that for my use cases, it might even make sense to make this the default behavior and add to add a flag for scanning all (including non-regular) files. That is more or less what we've been doing in-house at CCA when making extent statements from DFXML manifests.

Looking forward to hearing others' opinions and use cases.

Apologies for the many edits: that's what I get for writing this before I've finished my morning coffee :)

Cheers,
Tim

@richardlehane richardlehane added the bug label Sep 21, 2017

@richardlehane

This comment has been minimized.

Show comment
Hide comment
@richardlehane

richardlehane Sep 21, 2017

Owner

thanks Tim and Henk - yes agree that changing default to regular file scanning best approach. Will get a fix out soon

Owner

richardlehane commented Sep 21, 2017

thanks Tim and Henk - yes agree that changing default to regular file scanning best approach. Will get a fix out soon

@richardlehane

This comment has been minimized.

Show comment
Hide comment
@richardlehane

richardlehane Oct 4, 2017

Owner

This bug should now be resolved in latest release (sf 1.7.6). Please let me know if there are any further issues.

I have changed the default behaviour so that only "regular" files are now scanned (no symlinks, sockets, devices etc. - all these now report errors instead).

It may be worth adding back in the ability to scan symlinks in future through an optional flag (e.g. like how you can optionally follow symlinks with the file command - they have a flag for this). I haven't added this yet as unclear how this would work for symlinked directories (which currently cause sf to panic). But happy to explore if any demand for this.

Thanks again for the report Henk and thanks Tim for your input too

Owner

richardlehane commented Oct 4, 2017

This bug should now be resolved in latest release (sf 1.7.6). Please let me know if there are any further issues.

I have changed the default behaviour so that only "regular" files are now scanned (no symlinks, sockets, devices etc. - all these now report errors instead).

It may be worth adding back in the ability to scan symlinks in future through an optional flag (e.g. like how you can optionally follow symlinks with the file command - they have a flag for this). I haven't added this yet as unclear how this would work for symlinked directories (which currently cause sf to panic). But happy to explore if any demand for this.

Thanks again for the report Henk and thanks Tim for your input too

@hvanstappen

This comment has been minimized.

Show comment
Hide comment
@hvanstappen

hvanstappen Oct 4, 2017

hvanstappen commented Oct 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment