suggest output should be in VISL CG3 stream format

[flammie]

Like all the other steps in the typical grammarchecker pipeline, the last suggest step should default to linguist-readable VISL CG 3 format, i.e. same as divvun-suggest does:

$ cat tools/grammarcheckers/modes/smegram.mode
#!/bin/sh

hfst-tokenise -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/tokeniser-gramcheck-gt-desc.pmhfst' \
 | divvun-blanktag '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/analyser-gt-whitespace.hfst' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/valency.bin' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/mwe-dis.bin' \
 | cg-mwesplit \
 | divvun-blanktag '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/analyser-gt-errorwhitespace.hfst' \
 | divvun-cgspell -n 10 -b 15.000000 -w 5000.000000 -u 0.400000 -l '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/acceptor.default.hfst' -m '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/errmodel.default.hfst' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/valency-postspell.bin' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/grc-disambiguator.bin' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/spellchecker.bin' \
 | vislcg3 -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/grammarchecker.bin' \
 | divvun-suggest -g '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/generator-gramcheck-gt-norm.hfstol' -m '/Users/tpi006/github/giellalt/lang-sme/tools/grammarcheckers/errors.xml' -l se
$ echo 'cuolbma' | tools/grammarcheckers/modes/smegram.mode
"<cuolbma>"		cuolbma	→  čuolbma	→  čuolbmá	→  čuolmma	→  čuolbmi	→  čulbme	→  sulbmo	→  čuolbman	→  suolbmu	→  suolmmo	→  čuolbmal (msg: Čállinmeattáhus --- Hápmi ii leat sátnelisttus.)
	"čuolbma" N Sem/Dummytag Sg Nom <W:26.9993> <WA:11.9993> <spelled> "čuolbma"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Ind Prs Sg3 <W:35.3019> <WA:15.3019> <spelled> "čuolbmá"S SUGGESTWF &typo
typo
	"čuolbmái" A CmpN/SgN CmpN/PlG Sem/Hum Sg Acc <W:35.3019> <WA:15.3019> <spelled> "čuolbmá"S SUGGESTWF &typo
typo
	"čuolbmái" A CmpN/SgN CmpN/PlG Sem/Hum Sg Gen <W:35.3019> <WA:15.3019> <spelled> "čuolbmá"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Imprt ConNeg <W:41.6992> <WA:11.6992> <spelled> "čuolmma"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Imprt Sg2 <W:41.6992> <WA:11.6992> <spelled> "čuolmma"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Ind Prs ConNeg <W:41.6992> <WA:11.6992> <spelled> "čuolmma"S SUGGESTWF &typo
typo
	"čuolbma" N Sem/Dummytag Sg Gen <W:41.6992> <WA:11.6992> <spelled> "čuolmma"S SUGGESTWF &typo
typo
	"čuolbma" N Sem/Dummytag Sg Acc <W:41.6992> <WA:11.6992> <spelled> "čuolmma"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Imprt Du2 <W:45.3019> <WA:15.3019> <spelled> "čuolbmi"S SUGGESTWF &typo
typo
	"čuolbmat" V TV PrsPrc CmpNP/None <W:45.3019> <WA:15.3019> <spelled> "čuolbmi"S SUGGESTWF &typo
typo
	"čulbmet" V TV Ind Prs ConNeg <W:55.3019> <WA:15.3019> <spelled> "čulbme"S SUGGESTWF &typo
typo
	"čulbmet" V TV Ind Prs Sg3 <W:55.3019> <WA:15.3019> <spelled> "čulbme"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Ind Prt Pl3 <W:55.3019> <WA:15.3019> <spelled> "čulbme"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Ind Prs Du1 <W:55.3019> <WA:15.3019> <spelled> "čulbme"S SUGGESTWF &typo
typo
	"suolbmut" V IV Ind Prt Pl3 <W:55.3019> <WA:15.3019> <spelled> "sulbmo"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Actio Gen <W:58.0491> <WA:13.0491> <spelled> "čuolbman"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Actio Nom <W:58.0491> <WA:13.0491> <spelled> "čuolbman"S SUGGESTWF &typo
typo
	"čuolbmat" V TV Ind Prt ConNeg <W:58.0491> <WA:13.0491> <spelled> "čuolbman"S SUGGESTWF &typo
typo
	"čuolbmat" V TV PrfPrc <W:58.0491> <WA:13.0491> <spelled> "čuolbman"S SUGGESTWF &typo
typo
	"čuolbma" N Sem/Dummytag Ess <W:58.0491> <WA:13.0491> <spelled> "čuolbman"S SUGGESTWF &typo
typo
	"suolbmut" V IV Ind Prs Sg3 <W:60.3019> <WA:15.3019> <spelled> "suolbmu"S SUGGESTWF &typo
typo
	"suolbmut" V IV PrsPrc CmpNP/None <W:60.3019> <WA:15.3019> <spelled> "suolbmu"S SUGGESTWF &typo
typo
	"suolbmut" V IV Ind Prs ConNeg <W:60.3019> <WA:15.3019> <spelled> "suolmmo"S SUGGESTWF &typo
typo
	"čuolbmalit" V TV Gram/3syll Imprt ConNeg <W:60.3019> <WA:15.3019> <spelled> "čuolbmal"S SUGGESTWF &typo
typo
	"čuolbmalit" V TV Gram/3syll Imprt Sg2 <W:60.3019> <WA:15.3019> <spelled> "čuolbmal"S SUGGESTWF &typo
typo
	"čuolbmalit" V TV Gram/3syll Ind Prs ConNeg <W:60.3019> <WA:15.3019> <spelled> "čuolbmal"S SUGGESTWF &typo
typo
:\n

currently divvun-runtime says:

$ echo 'cuolbma' | divvun-runtime run -p tools/grammarcheckers/bundle.drb
[
  {
    "form": "cuolbma",
    "beg": 0,
    "end": 7,
    "err": "typo",
    "msg": [
      "Spelling error",
      "Not in the dictionary"
    ],
    "rep": [
      "čuolbma",
      "čuolbmá",
      "čuolmma",
      "čuolbmi",
      "čuolbman",
      "sulbmo",
      "suolbmu",
      "čulbme",
      "čuolbmal",
      "suolmmo"
    ]
  }
]

Having programmer-readable formats e.g. json in aiddition would also a cool option for any step though.

[Trondtr]

I want to support this bug message , but at the same time make it a bit more explicit, and relate it to present behaviour of divvun-runtime.

A tool like e.g. hfst-lookup can be used both interactively (command: hfst-lookup nob.hfst ENTER ja) or in a command (command echo ja | hfst-lookup nob.hfst). For divvun-runtime this seems to be the case as well. There is, however, a catch:

divvun-runtime run -p bundle.drb
Per prøver og gå.

gives 13 different outputs (if I count correctly). Then, on the other hand, the pipeline version:

е Per prøver og gå. | divvun-runtime run -p bundle.drb

gives only one (the 13th and last one).

We want:

1. the possibility of getting the same output of the pipeline version as we now have of the interactve version
2. the possibility of choosing what output to get. Usually we do not want 13 different outputs. Incicentally, what we actually want is not on the list of 13 modes, but a 14th one:

е Per prøver og gå.|sh modes/trace-nobgram-dev.mode 
"<Per>"
	"Per" N Prop Sem/Mal <W:0.0>
;	"per" Pr <W:0.0> REMOVE:895:per
: 
"<prøver>"
	"prøve" V Ind Prs <W:0.0> SELECT:11226
;	"prøve" N Fem Pl Indef <W:0.0> SELECT:11226
: 
"<og>"		og	→  å (msg: Bruk "å" i steden for "og" --- En infinitiv, som  "og", skal ha "å" foran seg, og ikke "og".)
	"og" CC <W:0.0> &real-og-aa ADD:1231:real-og-aa COPY:1241:real-og-aa
real-og-aa
	"å" <W:0.0> IM &real-og-aa &SUGGEST ADD:1231:real-og-aa COPY:1241:real-og-aa
å+IM	å
;	"og" Adv <W:0.0> REMOVE:13337
: 
"<gå>"
	"gå" V Inf <W:0.0>
;	"gå" V Imp <W:0.0> REMOVE:14337
"<.>"
	"." CLB <W:0.0>
:\n

Note that this presupposes --trace and the use of .cg3 and not .bin (which we want anyway).

Now, this 14th mode might be outside the scope of divvun-runtime, which is fine with us on the command line (as long as the modes folder is kept), but for the graphical divvun-runtime-playground adding this 14th mode would be the most used one. The two important things are : Do we get the sugestion (this is answered in mode 13 as well) and above all on what line number do I find the SELECT, ADD, COPY and ADDRELATION rules.

So:

• Having the 14th one both in the interactive and the pipeline version would be nice (and result in also us using the interacive one)
• For the pipeline version we waht a --output=trace, --output=14 etc., or however it is formulated (for all 14 modes)

[Trondtr]

... and yes: the og → å (or: WRONG → CORRECTED) part should colored

red → green

(the color disappeared, but should be visible)