Add a function to locate directories #2

Dietr1ch · 2018-01-26T08:31:31Z

It uses cd, locate and find as appropiate.

I'm still missing the binding, but it's not clear to me which one would be best, maybe we should just copy what fisherman/fzf does. What do you think?

gyakovlev · 2018-01-26T10:01:52Z

functions/fzy_select_directory.fish

+  set -l use_db "yes"
+  if command git rev-parse --show-toplevel >/dev/null ^/dev/null
+    set use_db "no"
+  else if command hg root >/dev/null ^/dev/null


t think there should be a check like

if which hg >/dev/null ^/dev/null and command hg root >/dev/null ^/dev/null

I'm not sure about logic after that, but it should check for hg first.
not checking git is fine, because it should be there to run fisherman in the first place.

otherwise I'm getting this trying to run without mercurial installed.

ya@reaper ~> fzy_select_directory fish: Unknown command 'hg' ~/.config/fish/functions/fzy_select_directory.fish (line 4): else if command hg root >/dev/null ^/dev/null ^ in function “__fzy_get_directories” called on line 57 of file ~/.config/fish/functions/fzy_select_directory.fish in function “fzy_select_directory” called on standard input >

gyakovlev · 2018-01-26T10:03:00Z

functions/fzy_select_directory.fish

+
+function fzy_select_directory
+  __fzy_get_directories | fzy | read -l foo
+  cd $foo


this cd here will put me back into ~ if I exit fzy search without selecting a directory (empty $foo). need to exit to cwd if $foo is empty.

gyakovlev · 2018-01-26T10:04:55Z

key_bindings.fish

@@ -1,3 +1,4 @@
 function fish_user_key_bindings
  bind \cr 'fzy_select_history (commandline -b)'
+  bind \ec 'fzy_select_directory'


bind \cf

like fzf does looks reasonable. not sure if it will ever clash with anything.

gyakovlev · 2018-01-26T10:16:22Z

functions/fzy_select_directory.fish

+    end
+  end
+
+  if not which locate >/dev/null ^/dev/null


I don't think using locate is a good idea.
I have directories sitting somewhere not indexed by locate, thus function with use_db=yes yields empty results.
need to check if this particular directory is present in the locate index (not sure if it's the right thing to do) or ditch locate completely.
or maybe create a setting to enable it unconditionally for people who have everything indexed.
another no-go for locate is that it's not always up to date and will return false results.

I added FZY_ENABLE_DB for allowing the use of locate

It uses `fd`, `locate` or `find` as appropiate. As locate database might have stale results, it's use must be enabled through `FZY_ENABLE_DB = "yes"` This function is bound to C-f

It really is a hack over locate and may lead to stale results. `fd` does fine when running on $HOME.

gyakovlev · 2018-01-28T03:42:38Z

hey, it's much cleaner and better with latest commits. let me test some edge cases and I'll merge it a bit later.
It's still kinda slow for large directories (like kernel source) but it's fine. I wonder if it's possible to use some fish non-blocking capability.
just going to /usr/src/linux and typing find . -type d | fzy is almost instant. but not with this function for some reason. I'll investigate later and see what can be done.

Dietr1ch · 2018-01-28T05:57:08Z

Hey give fd a try. It's smarter about skipping files and gets a nice speed advantage while retaining relevant results.

I can list "all" files on my home in 0.5s with fd, and 3.5s with find (find ran afterwards and might have some advantage). A 4x difference remains when dropping the output.

gyakovlev · 2018-01-28T07:25:05Z

I heard a lot of good things about fd, will give it a try sometime. thanks!

I've been trying to optimize the find to be somewhat faster and here is what I came up with.

find * -name ".*" -prune -o -type d -print ^/dev/null
I don't care about filtering fist 2 lines, because I'll rely on fzy finding the result.
also filtering /proc or anything else is not needed, I do sometimes have to search something in that fs.
can you test if on your side?

some points

-name is ~100x faster than -path for me.
don't call sed or tail, because clean relative path gets printed. just use the line.
fzy starts receiving find's output immediately in realtime
should be cross-platform and work everywhere.

even If I do cd / and hit ctrl+f it gives me results in a reasonable time (less than 3 second)
if I use it in any nested dir I do get results instantly. by the time I type anything fzy will receive rest of the data.

My tests may not reflect true picture, because my storage is ridiculously fast (nvme drives with 3.5 GB/sec read speed) so I need something or someone else to test it before I push.

I also see really good results with /usr/bin/printf "%s\n" $PWD/**/ but it does not start printing until it returns, so cannot use real-time pipe to fzy with it.

fun fact:

if you type any dir name into fzy prompt, even if it's not in the results, fish will cd into that dir on hitting Return, which is great!

ctrl+f is now my favourite keybindng, how did I live before this? it's so fast and convenient.

Dietr1ch · 2018-01-28T07:58:07Z

I just tried that on my root and noticed that it goes through remote mounts, it needs the -mount flag, it seems to be available on BSD's find too, so it should work on starbucks laptops too.
With that flag listing on my root takes 1.6s (a more modest ~550MB/s read sata ssd). This probably needs input from someone with an ancient rotating disk.

Also, it'll probably be much better to skip odd filesystems like proc, sysfs and others mentioned on mount.
I noticed that fd does badly /. Probably I'll fill a bug there to check the filesystem first.

gyakovlev · 2018-01-28T21:33:35Z

tried fd, it's good ) a bit slower than my find line, but insignificantly so. have no problems with it on /.

on the find function:
I find it REALLY useful for /sys, as sometimes I need to find that obscure dir in that jungle (something like devices/system/cpu/vulnerabilities or cgroup related). /proc is less useful but still fairly safe to search. especially if you mount /proc with hidepid=2. with hidepid=2 nothing you don't own gets shown in /proc.

another argument against -fstype is that it significantly slows down find. it runs several hundred times slower that way.
and even if you go to /sys | /proc to find something, find -fstype will refuse to search completely.

-mount skips mountpoints but not /proc or /sys, some people have a LOT of mounts.
on the other hand it makes sense to cd to possible target filesystem before searching in the first place.
I use a lot of zfs and btrfs montpoints and it fails in that case.

maybe requiring an additional input from a user is a sane option.
like hit enter to confirm search after ctrl+f if PWD is /

I'd really like to make it fast, because typical usage is just CDing withing ~ and slowing it down for some those edge cases makes me sad.

another possible problem like you mentioned is nfs/cifs/something else, slow as a turtle.

will test that new find line for a while, possibly with slow storage.
I had a old raspberry pi lying somewhere, I guess one with an external usb mechanical drive is slow enough with target to test.

gyakovlev · 2018-01-28T21:41:05Z

what do you think of this? pretty simple, works for /proc and /sys, only searches / mountpoint in single edge case, and still fast for regular lookups.

    function __fzy_get_directory_list -d 'Get a list of directories using fd or find'
      if which fd >/dev/null ^/dev/null
        command fd . -t d
      else if test "$PWD" = '/'
        command find / -mount -name ".*" -prune -o -type d -print ^/dev/null
      else
        command find * -name ".*" -prune -o -type d -print ^/dev/null
      end
    end

Dietr1ch · 2018-01-29T05:42:03Z

Yes, it makes sense to special-case for unusual places to let the default perform well on the common ones, although maybe we want to do that when outside ~/.
For the meantime I'd just try one that out to see how it behaves and then decide, worst case scenario we revert that change, but I think that at most it'll have adjustments as the idea seems fine to me.

I asked on fd repo and they told me to use --exclude to avoid network repos.
Maybe we want to ignore based on odd filesystems after parsing mount. I'll play with that later next week.

Dietr1ch force-pushed the directory_lookup branch 6 times, most recently from a76d442 to c90013f Compare January 26, 2018 09:00

gyakovlev requested changes Jan 26, 2018

View reviewed changes

gyakovlev added the enhancement label Jan 26, 2018

gyakovlev reviewed Jan 26, 2018

View reviewed changes

Dietr1ch force-pushed the directory_lookup branch from c90013f to b1aa28b Compare January 26, 2018 17:12

Add a function to locate directories

5ef3eae

It uses `fd`, `locate` or `find` as appropiate. As locate database might have stale results, it's use must be enabled through `FZY_ENABLE_DB = "yes"` This function is bound to C-f

Dietr1ch force-pushed the directory_lookup branch from b1aa28b to 5ef3eae Compare January 26, 2018 17:15

Dietrich Daroch added 2 commits January 27, 2018 15:26

select_directory: Add docstrings and use uniform quotes

f5b9e87

select_directory: Drop use of locate

b69e4cb

It really is a hack over locate and may lead to stale results. `fd` does fine when running on $HOME.

Dietr1ch force-pushed the directory_lookup branch from 68dda01 to b69e4cb Compare January 27, 2018 23:43

gyakovlev merged commit 84d4b87 into gyakovlev:master Jan 28, 2018

Dietr1ch deleted the directory_lookup branch January 28, 2018 05:58

Domon mentioned this pull request Jul 23, 2018

Ctrl-F conflicts with Fish's default #3

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a function to locate directories #2

Add a function to locate directories #2

Dietr1ch commented Jan 26, 2018 •

edited

gyakovlev Jan 26, 2018

Dietr1ch Jan 26, 2018

gyakovlev Jan 26, 2018

Dietr1ch Jan 26, 2018

gyakovlev Jan 26, 2018

Dietr1ch Jan 26, 2018

gyakovlev Jan 26, 2018

Dietr1ch Jan 26, 2018

gyakovlev commented Jan 28, 2018

Dietr1ch commented Jan 28, 2018

gyakovlev commented Jan 28, 2018 •

edited

Dietr1ch commented Jan 28, 2018

gyakovlev commented Jan 28, 2018

gyakovlev commented Jan 28, 2018

Dietr1ch commented Jan 29, 2018

Add a function to locate directories #2

Add a function to locate directories #2

Conversation

Dietr1ch commented Jan 26, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gyakovlev commented Jan 28, 2018

Dietr1ch commented Jan 28, 2018

gyakovlev commented Jan 28, 2018 • edited

fun fact:

Dietr1ch commented Jan 28, 2018

gyakovlev commented Jan 28, 2018

gyakovlev commented Jan 28, 2018

Dietr1ch commented Jan 29, 2018

Dietr1ch commented Jan 26, 2018 •

edited

gyakovlev commented Jan 28, 2018 •

edited