formula spec

[ickc]

Current YAML example

Formula

title: debug.py - inspect Pandoc's JSON output
author: Sergio Correia
homepage: https://github.com/sergiocorreia/panflute-filters
url: https://raw.githubusercontent.com/sergiocorreia/panflute-filters/master/filters/debug.yaml
version: 1.0.0
categories: [dev, inspect]
description: |
  This filter pretty prints the input it receives.
  This is useful when debugging panflute filters, or the package itself.
# install: <custom install commands besides simple & pip>
# uninstall: <nothing here>

Index

# Simple structure:
# - name: FILTER_NAME
#   url: FILTER_URL (optional if url-type is pip)
#   url-type: simple|pip (default is simple)

# Complex structure:
# - name: FILTER_NAME
#   branches:
#     branch1:
#       url: BRANCH_URL
#       url-type: simple|pip
#   url: FILTER_URL (optional if url-type is pip)
#   url-type: simple|pip (default is simple)

# Note: Branch 'default' will be run if no branch specified

- name: caps
  url: https://raw.githubusercontent.com/sergiocorreia/panflute/master/examples/panflute/caps.py
- name: debug
  url: https://raw.githubusercontent.com/sergiocorreia/panflute-filters/master/filters/debug.yaml
- name: tabulate-elements
  url: https://raw.githubusercontent.com/sergiocorreia/panflute-filters/master/filters/tabulate-elements.yaml
- name: pandoc-eqnos
  branches:
    - branch: default
      url-type: pip
    - branch: dev
      url: git://github.com/tomduck/pandoc-eqnos.git
      url-type: simple

# EOF

Proposed spec

From #2 (comment):

version: 1.0.0
license: MIT
summary: 'Some filter"
description: 'long description goes here'
author = xyz # or authors as a list
files: [xyz.py, abc.py] # This would just copy the files to the $datadir/filters folder
url: the url were the files are located
installer: pep # or cabal, etc. , this would run "pep install xyz" instead of copying files
homepage: 'https://github/someone/somerepo'

Get rid of index?

Should we get rid of index, or build index automatically from individual formula? Excerpt from #2 (comment):

concerning the index:

• Currently in the index, the non-simple kind has a complex structure (because if the package is non-simple, there's no obvious place to put a formula in the non-centralized formula "paradigm"). I think the complex structure should move to an individual formula now as the formulae are centralized.

• it means the url-type would be moved to individual formula as well, possibly renamed to/add a type to denote if it is a simple package (standalone, single file) or requires other package management (e.g. pip)

• since the formulae are centralized, url is not needed in the index.

After all these, it means the index is just a list of the names of packages. i.e. we might as well get rid of the index, or the index can be auto-generated, and then the index can simply be a plain text list of names, rather than in yaml.

concerning the individual formula, in addition to the existing

• the complex structure from the index

• url-type and/or type as said above

• more description on the non-simple filter. Currently, the only non-simple one is pip. It will be a problem for Unix users since the OSes ship with Python2 as the default, and it is considered a bad practice to override the default python version. Hence using pip alone means it will use pip2 for them. So an info on the python version of the package is needed: python2, python3, or univeral, corresponds to pip2, pip3, and pip.

  • This can be acheived by having a separate key to indicate the python version, or simply having the types as simple|pip|pip2|pip3.

[sergiocorreia]

If the only thing pandocpm does is allow installations, then you don't need the index file, as we can always check if the relevant yaml file exists.

If you want to allow searching (e.g. pandocpm search tables) then an index file is needed, but perhaps could be generated on the fly.

[ickc]

About searching:

It depends if we want to clone all formulae locally. If we do, then searching for tables means looking for tables.yml in any of the subfolders. I guess this is how homebrew did it.

But there are other reasons an index might be helpful. e.g. depending on the auto-filter for pandocfilters in #9 and formula spec in #7, if we have a key for the name of the "main" function in the pandocfilters (i.e. the name of that function can be arbitrary, e.g. "abc"), then we need a dict to quickly lookup for the name of the function when using auto-filter. In this case, the index format would better be not yaml but something builtin, say a dict in pkl, so that the autofilter can load it fast.

[ickc]

Another useful key will be something like "(in)compatibility" to indicate which other filter is incompatible with this. At the beginning this can be just an info that can be displayed or show in the gallery. In the future, auto-filters might emit warning msg if known pairs of incompatible filters are listed, for example.

[ickc]

I opened pandoc-extras/pandocpm-formulae which will be where we put the centralized formulae.

I also quickly gathered the existing formulae/spec we have discussed so far.

In the new structure the index reduces to very simple. (however, I'm thinking if it would be beneficial to define the "main" function name in the index too. Say someone write the "action" function as abc, and main: abc can be defined in the formula to let panflute points to that function instead. But I'm not sure if that will be too complicated)

I expanded the spec a bit in pandocpm-formulae/spec.yaml at master · pandoc-extras/pandocpm-formulae. Many of the keys are copied from setup.py, and many of them should not be mandatory but for auto-gallery only. There's a couple of questions in the comments, copied here:

# This would just copy the files to the $datadir/filters folder
# question: how would a single url point to 2 files? Does the URL suppose to be the path to `xyz.py` instead then?
# should they copied to `xyz` and `abc`, i.e. no extension?
# May be situation like these should be splited into 2 formulae?
files: [xyz.py, abc.py]

[ickc]

To expand my concern a bit:

I can see why one would have 2 filters under the same repository. But if a single formula installing both of them, would these 2 filters be really related in any way? Personally I do, pantable and pantable2csv, but is it common enough to justify this structure? e.g. I think I might split pantable and pantable2csv into 2 formulae even if such feature is given.

The downside of allowing this is, people might get lazy to submit different formula for different, unrelated filters. Then an auto-gallery won't be very helpful.

Of course we can still allow this feature but when there's pull request that includes unrelated filters this way, we can then tell them not to.