insert-directory: Listing directory failed but ‘access-file’ worked

[Qkessler]

Hi there!
I have built Emacs from source using your script, concretely one of your "Known good commits".
As is common with MacOS, after Catalina, there is a ton more security involved in folder handling, and for Emacs there is no exception. When trying to access any folder (dired really) using "Emacs.app", emacsclient or emacs commands, I get following error:
insert-directory: Listing directory failed but ‘access-file’ worked

The emacs --version command outputs:

GNU Emacs 28.0.50
Copyright (C) 2021 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

Following the subsequent resources, I have tried the following:

• Give full disk access to Emacs.app: reddit post
• Give full disk access to Ruby (both /usr/bin/ruby and /usr/local/opt/ruby/bin/ruby)

There is also Chris' post, but the workaround cannot be applied, considering we do not have any binary (besides Emacs) in the /Applications/Emacs.app/Contents/MacOS folder.

I would love some comment on this issue. It has surprised my not seeing any active issues on the matter in the repo, does anyone has had trouble with this?

[jimeh]

I can't say I've had this issue myself. If you haven't already, could you try adding your Emacs.app to the "Developer Tools" list under System Preferences > Security & Privacy > Privacy?

I had almost forgotten I had Emacs listed as a dev tool there cause I added it years ago, so potentially I've not run into this issue myself because of this.

[Qkessler]

Sadly I run into the exact same error.

At the moment, and for future reference, have tried, without luck:

• Add Emacs.app to "Developer Tools"
• Add Emacs.app to "Full disk Access"
• Add ruby (both versions: /usr/bin/ruby and /usr/local/opt/ruby/bin/ruby) to "Full Disk Access".

Best,

[jimeh]

Are you launching Emacs.app by double clicking on it in finder? Or are you launching it via terminal?

Is this error related specifically to viewing folders in dired which are backed by iCloud Drive, like Desktop/Documents/etc? or just any/all folders on the whole system?

Also the ruby stuff should be irrelevant cause we don't use a Ruby-based launcher script like builds from https://emacsformacosx.com does.

[Qkessler]

Thanks for the clarifications! I'm actually glad there is no real need to give Full disk Access to ruby, it feels wrong.
For some reason, commenting my (setq dired-listing-switches "-aBhl --group-directories-first") line did the trick. I imagine having Emacs.app with "Full Disk Access" is also key. They mention the problem here:
caldwell/build-emacs#84 (comment)

Thank you so much for the quick responses @jimeh!
Keep up the good work,
Best

[Qkessler]

Giving additional thought to the solution, the problem is within the --group-directories-first option. I imagine it would be interesting to give out a warning for this, instead of the generic, can't access dired, I can access the files.
Best

[jimeh]

Ah, interesting. I assume you're using using gls from the coreutils homebrew package to list files/folders in dired?

If so, does gls -aBhl --group-directories-first in the same directory yield any errors?

[Qkessler]

Nope, I wasn't using any additional gls package, just the plain old ls.

[jimeh]

Ah, that would do it. macOS' BSD-based built-in ls command doesn't have a --group-directories-first option, while the GNU-based ls command (exposed as gls by the coreutils formula) does support that flag.

Also worth mentioning is that the -B flag actually does different things between BSD and GNU ls implementations. Though it probably doesn't really matter much :)

[jimeh]

In case it's of any help, here is my personal dired setup to configuring the underlying ls command usage.

[Qkessler]

Thanks @jimeh for the detailed responses! Have a great end of the day!
Enrique.