Add Syntax_config

[mrnugget]

No description provided.

[mrnugget]

Let me show you the current state of this @rvantonder:

~/work/misc/custom_comby_matcher.json:

{
  "user_defined_delimiters": [
    [ "if", "fi" ],
    [ "BLOCK-BEGIN", "BLOCK-END" ]
  ],
  "escapable_string_literals": [],
  "escape_char": "\\",
  "raw_string_literals": [
    [ "\"", "\"" ]
  ],
  "comment_parser": [
    [ "Until_newline", "#" ]
  ]
}

Now I can use this to run comby like this:

$ ./comby -custom-matcher ~/work/misc/custom_comby_matcher.json 'BLOCK-BEGIN :[1] BLOCK-END' 'BLOCK-BEGIN nuked blocks BLOCK-END' -stdin << EOF
  BLOCK-BEGIN
    BLOCK-BEGIN
      block 1
    BLOCK-END

    BLOCK-BEGIN
      block 2
    BLOCK-END
  BLOCK-END
EOF

And the output is this:

------ /dev/null
++++++ /dev/null
@|-1,9 +1,1 ============================================================
-|  BLOCK-BEGIN
-|    BLOCK-BEGIN
-|      block 1
-|    BLOCK-END
-|
-|    BLOCK-BEGIN
-|      block 2
-|    BLOCK-END
-|  BLOCK-END
+|  BLOCK-BEGIN nuked blocks BLOCK-END

:)

---

What do you imagine the CLI to look like? Should users be able to specify a list of custom-matchers? A directory of them?

I think what I'd love to have is a way to connect a custom matcher with a file extension, so we could say ".foobar is handled by foobar-matcher, .monkey is handled by monkey matcher" etc.
The challenge here is to make the user interface clear. And since I haven't seen a good way of specifying name->value mappings on the CLI, how about something like this:

./comby -custom-matchers ~/matchers/scm.scheme.ss.json,~/matchers/monkey.mk.json,~/matchers/vimrc.vim.json

That would register 3 custom matchers:

• scm.scheme.ss.json for files with .scm, .scheme, .ss extensions
• monkey.mk.json for files with .monkey and .mk extensions
• vim.vim.json for files with .vimrc and .vim extensions

What do you think of that?

[rvantonder]

That sounds good to me! Initially I thought this could go in the JSON schema, but since the language extension is the only 'meta' data we need, I think your proposal works much better (and also easier to change).

The current solution would assume that we match substring extensions based on one dot separator (i.e., we won't support defining a custom matcher for a file extension like.foo.bar in file1.foo.bar versus foobar.bar in file2.foobar.bar). That's fine (indeed languages right now only use one extension), just making this explicit.

[rvantonder]

Also: yes, I think a comma-separated list on the CLI is a good way to specify multiple configurations (as opposed to pointing at a directory containing them). The latter could be supported later, but I think CLI is likely the more convenient obvious use case.

[rvantonder]

Aside: it's not currently possible to have a matcher for language 1 and a matcher for language 2 and run these on different languages in one execution. So supporting a comma-separated list doesn't make much sense until that is supported (and I'm not sure that that is a common use case). You can of course associate multiple file extensions with a single matcher, just pass it in the configuration after parsing: https://github.com/comby-tools/comby/blob/master/src/main.ml#L356.

[mrnugget]

Since we already have -filter-filters and only 1 matcher at a time is currently supported, the easiest would be to just specify --custom-matcher custom_scheme_matcher.json --file-filters scm,scheme,ss. Right?

Then the only thing left to do would be to update the documentation for file-filters to say that it also works with --custom-matcher.

I actually really like this, because it keeps every piece simple and they all work together. What do you think?

[mrnugget]

Update: that already works really well. Here's an example of using a custom matcher with a file filter:

./comby\
  -custom-matcher ~/work/misc/custom_comby_matcher.json\
  -f .custom_block
  'BLOCK-BEGIN :[1] BLOCK-END'\
  'BLOCK-BEGIN nuked blocks BLOCK-END'

------ /Users/thorstenball/work/comby/example.custom_block
++++++ /Users/thorstenball/work/comby/example.custom_block
@|-1,9 +1,1 ============================================================
-|BLOCK-BEGIN
-|  BLOCK-BEGIN
-|    block 1
-|  BLOCK-END
-|
-|  BLOCK-BEGIN
-|    block 2
-|  BLOCK-END
-|BLOCK-END
+|BLOCK-BEGIN nuked blocks BLOCK-END

[rvantonder]