Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement] Search for filters/lua-filters within sub-directories of $DATADIR/filters #6635

Open
the-solipsist opened this issue Aug 24, 2020 · 4 comments · May be fixed by #8822
Open

[Enhancement] Search for filters/lua-filters within sub-directories of $DATADIR/filters #6635

the-solipsist opened this issue Aug 24, 2020 · 4 comments · May be fixed by #8822
Labels
lua-filters out of scope? status:more-discussion-needed

Comments

@the-solipsist
Copy link
Contributor

the-solipsist commented Aug 24, 2020 •
edited

Problem:
Currently, if a lua-filter (say wordcount.lua) is at $DATADIR/filters/wordcount.lua, things work fine. But if it is in $DATADIR/filters/wordcount/wordcount.lua, then the pandoc command is unable to locate it.

Commands:

$ cd $HOME/.local/share
$ git clone https://github.com/pandoc/lua-filters.git pandoc/filters
$ pandoc --lua-filter=wordcount.lua ~/example.md

Expected:

311 words in body
1741 characters in body
2014 characters in body (including spaces)

Actual Output:

Error running filter wordcount.lua:
wordcount.lua: openBinaryFile: does not exist (No such file or directory)

Workaround:

$ pandoc --lua-filter=wordcount/wordcount.lua ~/example.md

The relevant code seems to be in this file:

pandoc/src/Text/Pandoc/Filter/Path.hs

Lines 22 to 35 in 93e3d46

expandFilterPath :: PandocMonad m => FilePath -> m FilePath
expandFilterPath fp = do
mbDatadir <- getUserDataDir
fpExists <- fileExists fp
if fpExists
then return fp
else case mbDatadir of
Just datadir | isRelative fp -> do
let filterPath = datadir </> "filters" </> fp
filterPathExists <- fileExists filterPath
if filterPathExists
then return filterPath
else return fp
_ -> return fp

Please close this issue if it isn't worth changing, since the "workaround" works well enough. However, some software (for instance prosody wrt to it modules) would allow you to refer to "wordcount/wordcount.lua" as "wordcount" (dropping both the directory name as well as .lua, as that's the extension it would be looking for).
@argent0
Copy link
Contributor

argent0 commented Sep 10, 2020

Hi, what would happen If you have two files with the same name under different sub-folders ?

@the-solipsist
Copy link
Contributor Author

the-solipsist commented Sep 16, 2020 •
edited

Hi, @argent0. Prosody uses very simple logic: if you ask it to load {modulename} it either looks for {modulename}/{modulename}.lua or {modulename}.lua within the module paths that you've set[1]. So, something similar can be done here too, given that the the lua-filters git directory is also organized as {filtername}/{filtername}.lua.

So, to answer your question, as per my proposed logic, Pandoc would only recognize a file if it is either in the base directory, or in a sub-directory with the same name as the file.

[1]: For simplicity's sake, I'm discounting the optional "mod_" that Prosody uses for identifying modules. For the exact code, see the link I've provided to "pluginloader.lua".

@tarleb
Copy link
Collaborator

tarleb commented May 6, 2021

I'm sympathetic with the request, it would make using the lua-filters repo much easier. However, I'm wary of the complexity that comes with such a feature, so I lean towards treating it as out of scope for pandoc.

@pbsds pbsds mentioned this issue Feb 14, 2022
Open
tarleb added a commit to tarleb/pandoc that referenced this issue May 4, 2023
@tarleb
PandocMonad: allow nested data files
071e58e 
Data file fallbacks are now also searched in the directory that has the
same name as the file minus the extension. For example, a filter with
name `frob.lua` will be searched in `$DATADIR/filters/frob.lua` as well
as in `$DATADIR/filters/frob/frob.lua`.

This allows to place full git repositories in the data directory.

Closes: jgm#6635
@tarleb tarleb linked a pull request May 4, 2023 that will close this issue
Open
@tarleb
Copy link
Collaborator

tarleb commented May 4, 2023 •
edited

I've changed my stance on this and opened #8822. My reasoning is that I'd rather see people use symbolic links to the filter file; however, Windows doesn't work that well with symbolic links, and even on posix it's often convenient to simply place a fulle git subtrees in the data dir.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lua-filters out of scope? status:more-discussion-needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

PandocMonad: allow nested data files tarleb/pandoc
3 participants
@tarleb @argent0 @the-solipsist