Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WalkParallel does not show unaccessible directories. #1346

Closed
treere opened this issue Aug 9, 2019 · 1 comment
Closed

WalkParallel does not show unaccessible directories. #1346

treere opened this issue Aug 9, 2019 · 1 comment
Labels
bug A bug.

Comments

@treere
Copy link

treere commented Aug 9, 2019

If a directory is not accessible the dta produced by WalkParallel is different from the one created by WalkDir: the unaccessible directory is not listed.

How to reproduce:
Prepare the env (I use Linux)

# create dir 
mkdir imnotlisted
# make it unaccessible
sudo chmod 700 imnotlisted
sudo chown root:root imnotlisted 

Using the small programm below the parallel version

./small_program parallel imnotlisted

outputs:

Err(WithDepth { depth: 0, err: WithPath { path: "iminvisible", err: Io(Os { code: 13, kind: PermissionDenied, message: "Permission denied" }) } })

but the sequential version

./small_program walkdir imnotlisted

outputs

Ok(DirEntry { dent: Walkdir(DirEntry("iminvisible")), err: None })
Err(WithPath { path: "iminvisible", err: Io(Custom { kind: PermissionDenied, error: Error { depth: 0, inner: Io { path: Some("iminvisible"), err: Os { code: 13, kind: PermissionDenied, message: "Permission denied" } } } }) })

Which is the desired behaviour? In ripgrep this is not a problem.


the "small program below"

extern crate crossbeam_channel as channel; 
extern crate ignore; 
 
use std::env; 
use std::thread; 
 
use ignore::WalkBuilder; 
 
fn main() { 
    let mut path = env::args().nth(1).unwrap(); 
    let mut parallel = false; 
    let (tx, rx) = channel::bounded::<Result<ignore::DirEntry,ignore::Error>>(100); 
    if path == "parallel" { 
        path = env::args().nth(2).unwrap(); 
        parallel = true; 
    } else if path == "walkdir" { 
        path = env::args().nth(2).unwrap(); 
    } 
 
    let stdout_thread = thread::spawn(move || { 
        for dent in rx { 
            println!("{:?}",dent); 
        } 
    }); 
 
    if parallel { 
        let walker = WalkBuilder::new(path).threads(6).build_parallel(); 
        walker.run(|| { 
            let tx = tx.clone(); 
            Box::new(move |result| { 
                use ignore::WalkState::*; 
 
                tx.send(result).unwrap();
                Continue
            }) 
        }); 
    } else { 
        let walker = WalkBuilder::new(path).build(); 
        for result in walker { 
            tx.send(result).unwrap(); 
        } 
    } 
    drop(tx); 
    stdout_thread.join().unwrap(); 
} 

(edit: fixed small program...was all lowercase)

@BurntSushi
Copy link
Owner

I think the sequential version is the expected behavior, since there's no problem reading the directory entry from its parent, right? It's only descending into it that results in an error.

I'm not sure when I'll look into this, but a PR is welcome.

@BurntSushi BurntSushi added the bug A bug. label Aug 9, 2019
BurntSushi pushed a commit that referenced this issue Feb 17, 2020
This commit fixes an inconsistency between the serial and the parallel
directory walkers around visiting a directory for which the user holds
insufficient permissions to descend into.

The serial walker does produce a successful entry for a directory that
it cannot descend into due to insufficient permissions. However, before
this change that has not been the case for the parallel walker, which
would produce an `Err` item not only when descending into a directory
that it cannot read from but also for the directory entry itself.

This change brings the behaviour of the parallel variant in line with
that of the serial one.

Fixes #1346, Closes #1365
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants