Ability to view/index hidden files

[lasers]

I discovered this gem today. I loved it. I wanted to know how I can add hidden directories / files to the indexing. That way, I can open the configuration files (with preferred/different editor) sometimes without using the terminal. Can we support this option? I do want the option to view/index the entire home directory if that's possible.

[MarkHedleyJones]

Hi Lasers, thanks for the feedback.
I have made the following changes which should suit your use case.
Added three boolean items to the preferences file:

• "scan_hidden_folders" Set to true to allow the scanner to trawl through hidden folders.
• "include_hidden_files" Set to true to include hidden files found by the scanner.
• "include_hidden_folders" Set to true to include hidden folders the scanner finds.

Added support for the following two entries in "valid_extensions"

• "" Will include any file that has no extension.
• "*" Will include all files.

So if you want to view/index the entire home directory. You'll want the relevant parts of you preferences file to look like:

"scan_hidden_folders": true,
"include_hidden_files": true,
"include_hidden_folders": true,
"valid_extensions": ["*"]

[lasers]

Mark! Thank you very much for implementing this! You just made my
dmenu-extended much simpler. I don't need to worry about extensions anymore
too! It was only a little hassle in maintaing the extension list to ensure
that I at least include mp4, cbz, cbr, epub, et cetera... The list goes on!

I'll get everything out in the open right away. I'm surprised to find that
this package didn't garner enough attention. I gave it a star and a vote on
Archlinux AUR. ;)

I think it could be very benefiting to make DuckDuckGo a default search
engine. That way, when people search, they don't see "Change Provider"
repeatedly... but rather to something like "Search (DuckDuckGo):" along
with a list.

DuckDuckGo got a nice bang thing going on.
https://duckduckgo.com/bang.html

"Search (DDG): !yt ign review resistance 3" would fetch me a direct link to
a result on YouTube.

If one were to select a different search engine, one would receive "Search
(Google Image):"

New question. I have a personal bin script (eg, ~/.bin, ~/bin) . Wallpaper
changer and pipes and whatnot. Is it possible for dmenu-extended to
recognize all executable binaries inside $PATH?

Scratch that. I found the answer (filter_binaries) long after I typed up
the question above. Just being verbose right now. Would it be a good idea
to have an option to turn off "sort by characters" (Back to ABC) just like
dmenu? It's for those who might want to have their filter_binaries off.

If I have any more questions/suggestions, I will gladly run them by you.
I'm trying not to bloat your project with endless features. :)

On Sun, Sep 21, 2014 at 6:36 PM, Mark H. Jones notifications@github.com
wrote:

Hi Lasers, thanks for the feedback.
I have made the following changes which should suit your use case.
Added three boolean items to the preferences file:

• "scan_hidden_folders" Set to true to allow the scanner to trawl
  through hidden folders.
• "include_hidden_files" Set to true to include hidden files found by
  the scanner.
• "include_hidden_folders" Set to true to include hidden folders the
  scanner finds.

Added support for the following two entries in "valid_extensions"

• "" Will include any file that has no extension.
• "*" Will include all files.

So if you want to view/index the entire home directory. You'll want the
relevant parts of you preferences file to look like:

"scan_hidden_folders": true,
"include_hidden_files": true,
"include_hidden_folders": true,
"valid_extensions": ["*"]

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[lasers]

Hi again, I was wrong about $PATH because I was experimenting with a +script. ​I'm unable to launch scripts from dmenu-extended unless I +$NAME_OF_THE_SCRIPT. I feel like it would be a good plugin (to support listing either bash/zsh aliases/functions in a submenu)? Some of them are bound to be GUI-based.

[MarkHedleyJones]

Lasers, you have given me much to think about - thank you for your input.
I have updated dmenu-extended to recognise all folders in the system path which should include your ~/.bin, ~/bin folders.
I will consider your suggestion for DuckDuckGo, it certainly wont harm adding it to the search engine plugin. I agree that it might be a more suitable default due to its enhanced privacy. The bang feature is pretty cool.
You are the second person to request a plugin to support aliases so this is something I will need to make progress on. I haven't used alias files myself so any advice on specific implementations are welcome.

[lasers]

Sorry. I still can't run the scripts inside my $PATH. I like this project.
It's amazing.... It could be a dmenu-based synapse (a semantic file
launcher) for tiling managers. I noticed this (minor?) annoyance right away.

Rebuilding caches take time (too long for me) and I'm forced to either wait
it out (or to send it to background ; but with no way of acknowledging when
it's done). Would it be a good idea to have it automatically building a
cache in background and have notify-send sending out a "Cache updated
successfully" message? I'm just brainstorming this particular issue.

I also thought about avoiding submenu because it's not searchable unless
you go through with the option. It's not 100% intuitive like
https://wiki.ubuntu.com/Unity/HUD

δε:
Search -> DuckDuckGo (ddg)
Search -> Github (gh)
Search -> Google (g)
Search -> Wikipedia (w)
Options -> Download new plugins
Options -> Edit menu preferences
Options -> Rebuild cache
Options -> Remove existing plugins
Options -> Update installed plugins
Chromium
Dolphin Emulator
ePSXe
Firefox
Google Chrome
Google Chrome (beta)
htop
Kega Fusion
Minetest
Oracle VM VirtualBox
PCSX2
Sublime Text 3 Dev
Xonotic
Depression Quest
LibreOffice
LibreOffice Calc
LibreOffice Impress
LibreOffice Writer
Thunar
Zathura
2to3
2to3-2.7
2to3-3.4
7z
7zFM
7zG
7za
7zr
/home/chris/Books
/home/chris/Documents
/home/chris/Downloads
/home/chris/Games
/home/chris/Library
/home/chris/Music
/home/chris/Pictures
/home/chris/Videos
/home/chris/VirtualBox VMs
/home/chris/src

I'm listing them in this order.

• Search
• Options
• Desktop Apps
• Binaries
• Files & Directories

As for dmenu to support aliases/functions, I don't know what to suggest other than to take a look at aur/dmenu-aliases. https://github.com/orschiro/orschiro-pkgbuilds/tree/master/dmenu-aliases

You're welcome to shut down my suggestions/feature requests.

[MarkHedleyJones]

Ill have a look into the $PATH issue again. Just to confirm, when you type

echo $PATH

into a shell do you get the relevant paths (those in your home directory) listed? Also, have you pulled the latest version from git?

There is a message that pops up when the cache has been rebuilt but it must not be triggering on your system - something else I will look into. The message says how many items are in the cache and by how much the size changed from the rebuild. I noticed it wasn't appearing on Ubuntu systems. I'll update here when I've sorted this.

Your ordering is something I've played around with a bit in the past. The reason I sort by length is so that if I have the following items in my cache and I enter music I want to see:

/home/mark/Music/
/home/mark/Downloads/Music
/home/mark/Archive/Musical_Instruments

and not

/home/mark/Archive/Musical_Instruments
/home/mark/Downloads/Music
/home/mark/Music/

However I just tried this again with an alphabetically sorted cache and this doesn't seem to be a problem so perhaps when I made the decision to sort by length I was using a pre 4.5 version of dmenu where the built-in sorting is not as good. Either way changing the sort order is now a possibility so I will put some thought into how this might be defined in the configuration.

I also thought about avoiding submenu because it's not searchable unless you go through with the option.

Good point. My personal preference is to keep the cache clean; tucking options away was a natural move. I also like it this way so that plugins will have their own menus and there is no way they may conflict with each other. However I'm a big fan of the way Sublime Text does things, especially the command palette which is laid out as you have suggested. Also it utilises the power of dmenu to locate where the relevant options are, removing the need to know where the items are.

I guess to some extent I have shied away from allowing plugins to directly add items to the cache, instead leaving all plugin functionality inside the relevant menu. It may be time to add a way for plugins to modify the cache directly.

Thanks for the link to dmenu-aliases, this should help.

So, takeaway points for me are:

• Cache sorting and sub-cache ordering should be configurable via preferences.
• Plugins should be able to add/remove items from the cache/caches.
• Nesting options in submenus or flattening the plugin options should be controlled from preferences and probably flattened by default.

Aliases need to be implemented better. But what exactly should I do?
1 - Scrap the built in alias function
2 - Offer alias file parsing as an option in preferences
4 - Offer alias file parsing as a plugin
8 - Auto-detect alias file and include by default

My gut feeling is 5 (4+1).

P.S. Thanks for posting your thoughts here by the way. It's good having someone to bounce ideas off.

[lasers]

Hi Mark,

$ echo $PATH

/home/chris/.local/bin:/home/chris/.bin/:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

I was abusing dmenu-extended looking for my scripts to appear in the list when you said you added the feature. (I upgraded the package too). It's working quite fnow. I may have forgetten to either turn off the filter_binaries (and/or) rebuild caches. I'm not so sure anymore. I kept stuffs in ~/.local/bin.

Also, have you pulled the latest version from git?

I'm using this package. https://aur.archlinux.org/packages/dmenu-extended/

Yet More Suggestions.... The features on https://kien.github.io/ctrlp.vim expresses things (as well as possibitlies) for dmenu-extended quite well.

• Add option to ignore extensions (?)

*.exe, *.iso, *.so, *.swp, *.zip    # ... and more

• Add option to ignore glob directories (?)

*.git, *.hg, *.svn, *cache*, *.dll, *.so, *.DS_Store   # ... and more

/home/chris/.config/dmenu-extended/cache/dmenuExtended_main.txt
/home/chris/.config/dmenu-extended/cache/dmenuExtended_plugins.txt
... wouldn't show up in list either. ;)

• Add option to filter_common_folders (?)
  This is to weed out unavoidable directories that might have either a single innocent-looking file or hundred files that had little or no signifcant difference to us at this time.)

~/.adobe ~/.dbus ~/.cache ~/.local/share ~/.local/share/Trash
~/.macromedia/ ~/.gstreamer-0.10 ~/.thumbnails ~/.pki    # ... and more

I looked into ~/.local/share (and it's for apps to interact with the data) and not the directory that the users might want to access often. Important data get saved there -- but not important enough to be crawled for on-off access/editing. Thoughts? https://askubuntu.com/questions/14535/whats-the-local-folder-for-in-my-home-directory

Maybe I'm being riduclous with the filtering_common_folders option when I can turn off include_hidden_*. I think I am. I'm trying to supply you with nice features. Options to exclude glob directories and extensions sounds reasonable, but I'm not so sure about the this next one.

• Add option to filter_cache (?)
  Am I being bewilderingly silly for suggesting several additions/filters/optimization for quick on-off lightweight dmenu usage? God. I hope not. I mean, I'm only looking at cache at this moment. There are lot of cache and this mean lot of unnecessary things for dmenu-extend to parse through... (especially when you're crawling everything in the house). I think I'm doing this because I'm scrolling through sorted-by-length.

Thanks bleachbit for this. With filter_cache "true", it could parse through bleachbit's cleaner/*.xml to filter out the caches resulting in a nicer/minimal dmenu-extended experience... especially with hidden folder/files enabed. Definitely a plugin. http://sprunge.us/HPXK

P.S. Check out deepscan.xml -- It covers .DS_Store, Thumbs, backups, et cetera.

• Avoid dmenuExtended_ inside dmenu-extended directory (?) (Meh)

/home/chris/.config/dmenu-extended
/home/chris/.config/dmenu-extended/cache
/home/chris/.config/dmenu-extended/config
/home/chris/.config/dmenu-extended/plugins
> /home/chris/.config/dmenu-extended/cache/dmenuExtended_main.txt
> /home/chris/.config/dmenu-extended/cache/dmenuExtended_plugins.txt
> /home/chris/.config/dmenu-extended/config/dmenuExtended_preferences.json
/home/chris/.config/dmenu-extended/config/internetSearch.json
/home/chris/.config/dmenu-extended/plugins/__init__.py
/home/chris/.config/dmenu-extended/plugins/__pycache__

• Customized search algorithm (?)

Your ordering is something I've played around with a bit in the past. The reason I sort by length is so that if I have the following items in my cache and I enter music I want to see:

I'll show you why I found it a problem to have lines sorted by length. I made three things
Sorted by length, sorted by ABC's, sorted by customized search algorithm.
http://sprunge.us/dhdS

I hope you like them! I enjoy testing stuffs.

[MarkHedleyJones]

I've made some progress towards implementing most of what we've discussed in the last three posts. Quite a lot of changes are happening, especially to the preferences. These are sitting in the new_scanning branch. Feel free to test it and let me know what you think but be warned - there will be some rough edges.
If you do try, rename your configuration so a new one will be generated but keep your existing one. You may need to remove the internet search and system package management plugins before trying as I haven't tested it with them yet.

These are some of the options available:

"include_files": ['*'],
"exclude_files": [],
"include_folders": ['~/'],
"exclude_folders": [],
"include_hidden_files": True,
"include_hidden_folders": True,
"follow_symlinks": False,
"group_order": {
    "plugins": 0,
    "applications": 1,
    "binaries": 2,
    "files": 4,
    "folders": 3,
    "aliases": 5
},
"group_sort_method": {
    0: 'length',
    1: 'length',
    2: 'length',
    3: 'length',
    4: 'alpha',
    5: 'length'
},
"alias_files": [], # TODO
"exclude_application_binaries": True,
"pluginItems_indicator_nested": "-> ",
"pluginItems_indicator_flat": ": ",
"pluginItems_display": 'flat',

The "group_order" variable sets how the items are ordered. Many items may share the same number so they are sorted together.

"group_sort_method" allows you to select the sort method for each of the groups defined in "group_order", with possible values of "length" or "alpha".

"include_files" and "exclude_files" will control the cache. You have the option to set "*" for "include_files" to include everything and then set patterns in "exclude_files" and "exclude_folders" to remove sections.

"exclude_files" can be configured as you mentioned earlier with

*.exe, *.iso, *.so, *.swp, *.zip # ... and more

"exclude_folders" can be configured as you mentioned earlier with

.git, *.hg, *.svn, *cache, *.dll, *.so, *.DS_Store # ... and more

and also with:

~/.adobe ~/.dbus ~/.cache ~/.local/share ~/.local/share/Trash ~/.macromedia/ ~/.gstreamer-0.10 ~/.thumbnails ~/.pki # ... and more

"pluginItems_display" sets whether all the plug-in items are displayed in the list or nested in their own menu. This enables the following plugin menu layout:

Options: Download new plugins
Options: Edit menu preferences
Options: Rebuild cache
Options: Remove existing plugins
Options: Update installed plugins

And I'm working towards being able to specify the location of an alias file in the preferences, which is what "alias_files" will do.

Your thoughts are welcome, even if you don't get a chance to test.

[lasers]

Facing rough edges...

• Running from "new_scanning" branch with clean profile. == No menu! (Open)
• Running from "master" branch with clean profile. == Yes menu! (Building cache)

How do I forward from here? I looked at both (default) config and I noticed several changes so I'm asking you since I don't know what to merge. It might be something you need to work on? I can't test any other new features because I'm stumped above. I'll wait for your support / confirmation first.

I also have a wishful feature request. To add an (+) indicator for manually added entries so I can differentiate between binaries + manually added entries.

Added: I see I could do something like +input-command # input-same-command (with the power of alias indicator). It may or may not be quite the same thing -- but I can work with that by giving them names (instead of empty manually added entries). It would still be nice to have an indicator though.

[MarkHedleyJones]

Sorry about that. It should generate the cache now. Aliases and manually included items are still not working but configuration of menu organisation is functional (try changing "pluginItems_display" to 'nested' and rebuild the cache)

As for your feature request, I am considering merging manually added items and aliases together since they are essentially the same thing.

I haven't thought that part through properly yet. Perhaps moving 'aliases' and 'manually added entry' functionality to plugins is a better idea, each having its own configuration.

[lasers]

Hi. Do you IRC? I'll be in #dmenu on freenode for now.

• Clean profile (from the terminal)

chris@xps ~ $ i3sh --dmenu-extended         
Setting up dmenu-extended prefs files...
Plugins directory created
Cache directory created
prefs directory created
Preferences file created at: /home/chris/.config/dmenu-extended/config/dmenuExtended_preferences.json
Traceback (most recent call last):
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 464, in cache_save
    f.write(item+"\n")
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc94' in position 49: surrogates not allowed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended_run", line 11, in <module>
    dmenu_extended.run(debug)
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 1037, in run
    cache = d.cache_load()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 525, in cache_load
    self.cache_regenerate()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 448, in cache_regenerate
    cache = self.cache_build()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 750, in cache_build
    self.cache_save(tmp, path_cache + '/dmenuExtended_group' + str(level) + '.txt')
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 491, in cache_save
    f.write(item+'\n')
TypeError: 'str' does not support the buffer interface

• Sometimes when I had working dmenu-extended, I noticed quirks. So I ran it from the command this time.

chris@xps ~ $ i3sh --dmenu-extended
Setting up dmenu-extended prefs files...
Plugins directory exists - skipped
Cache directory exists - skipped
prefs directory exists - skipped
Existing preferences file found, will not overwrite.
Traceback (most recent call last):
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended_run", line 11, in <module>
    dmenu_extended.run(debug)
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 1037, in run
    cache = d.cache_load()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 525, in cache_load
    self.cache_regenerate()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 448, in cache_regenerate
    cache = self.cache_build()
  File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 664, in cache_build
    for item in plugin['plugin'].menu_items():
AttributeError: 'extension' object has no attribute 'menu_items'

I tried rearranging group_order (plugins, aliases, applications, binaries, files, folders, plugins). That didn't happen. Alpha did not work for me either. No problem reporting my findings.

[MarkHedleyJones]

Hi again and thanks for testing. No I don't use IRC but may give it a try
in future. I have been quite busy over the last week and will be next week
too so haven't been getting much time to work on on the menu. But I'll be
back!
The second error is because the new branch isn't compatible with the
existing plugins yet.
The first two errors concern me. I've never encountered them before. What
distribution and Python version do you use?

Mark
On 1/10/2014 5:14 pm, "lasers" notifications@github.com wrote:

Hi. Do you IRC? I'll be in #dmenu on freenode for now.

• Clean profile (from the terminal)

chris@xps ~ $ i3sh --dmenu-extended
Setting up dmenu-extended prefs files...
Plugins directory created
Cache directory created
prefs directory created
Preferences file created at: /home/chris/.config/dmenu-extended/config/dmenuExtended_preferences.json
Traceback (most recent call last):
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 464, in cache_save
f.write(item+"\n")
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc94' in position 49: surrogates not allowed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/chris/src/_git/dmenu-extended/dmenu_extended_run", line 11, in
dmenu_extended.run(debug)
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 1037, in run
cache = d.cache_load()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 525, in cache_load
self.cache_regenerate()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 448, in cache_regenerate
cache = self.cache_build()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 750, in cache_build
self.cache_save(tmp, path_cache + '/dmenuExtended_group' + str(level) + '.txt')
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 491, in cache_save
f.write(item+'\n')
TypeError: 'str' does not support the buffer interface

• Sometimes when I had working dmenu-extended, I noticed quirks. So I
  ran it from the command this time.

chris@xps ~ $ i3sh --dmenu-extended
Setting up dmenu-extended prefs files...
Plugins directory exists - skipped
Cache directory exists - skipped
prefs directory exists - skipped
Existing preferences file found, will not overwrite.
Traceback (most recent call last):
File "/home/chris/src/_git/dmenu-extended/dmenu_extended_run", line 11, in
dmenu_extended.run(debug)
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 1037, in run
cache = d.cache_load()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 525, in cache_load
self.cache_regenerate()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 448, in cache_regenerate
cache = self.cache_build()
File "/home/chris/src/_git/dmenu-extended/dmenu_extended.py", line 664, in cache_build
for item in plugin['plugin'].menu_items():
AttributeError: 'extension' object has no attribute 'menu_items'

I tried rearranging group_order (plugins, aliases, applications, binaries,
files, folders, plugins). That didn't happen. Alpha did not work for me. No
problem reporting my findings.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[lasers]

Archlinux.
Python 3.4.1 (default, May 19 2014, 17:23:49)
dmenu-4.5, © 2006-2012 dmenu engineers, see LICENSE for details