Recognize non-ASCII punctuation chars

[SilverRainZ]

The punctuation_chars.h header file is auto-generated from gen_punctuation_chars.py.
I also add a test case "Unicode Punctuation Chars":

before:

  inline_markup:
    ✗ Unicode Punctuation Chars

1 failure:

correct / expected / unexpected

  1. Unicode Punctuation Chars:

    (document
      (paragraph)
      (paragraph)
      (paragraph)
      (paragraph))
      (paragraph
        (emphasis))
      (paragraph
        (emphasis)
        (strong))
      (paragraph
        (emphasis))
      (paragraph
        (emphasis)))

after:

  inline_markup:
    ✓ Unicode Punctuation Chars

Any comments are welcome.

Close #53.

[SilverRainZ]

@stsewd Can you please review it?

[stsewd]

@SilverRainZ thank you for opening this PR! I'll try to take a look at it this weekend or the next one (sorry, busy weeks). I just noticed that the Windows CI is failing with this change.

[SilverRainZ]

The Windows CI failed with a weird error message:

 scanner.c
D:\a\tree-sitter-rst\tree-sitter-rst\src\tree_sitter_rst\punctuation_chars.h(107,41): error C2059: syntax error: '}' [D:\a\tree-sitter-rst\tree-sitter-rst\build\tree_sitter_rst_binding.vcxproj]
  (compiling source file '../src/scanner.c')

I have checked L107 and there is nothing special, I have no idea for now.

// ...
  L'\u201f',
};
const int32_t start_chars_range[][2] = {}; // <-- L107

const int32_t delim_chars[] = {
// ...

[SilverRainZ]

It seems that we should update the WASM binary after any changes, but I think it should be done with the maintainer.

B.T.W, the npm run wasm is broken due to tree-sitter/tree-sitter#3202.

[stsewd]

Thanks for this great contribution! I did some small edits and fixed the Windows build by not generating empty arrays (since the array was empty, when trying to access to the first element, it was probably pointing to invalid memory/values, and Windows didn't like that).