Welcome to aGrUM Discussions!

[phwuil]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[mingodad]

I noticed that this project uses CocoR parser generator and I did several improvements/fixes to it here https://github.com/mingodad/CocoR-CPP , https://github.com/mingodad/CocoR-CSharp , https://github.com/mingodad/CocoR-Java and also recently a port to Typescript/Javascript here https://github.com/mingodad/CocoR-Typescript with an online playground here https://mingodad.github.io/CocoR-Typescript/playground .

I think that they can be useful to this project and I'm sharing it here.

[mingodad]

Just out of curiosity I translated this project grammars to the EBNF understood by https://www.bottlecaps.de/rr/ui to generate navigable railroad diagrams (https://en.wikipedia.org/wiki/Syntax_diagram) and it seems that some of then seem overkill to use a parser generator.

Copy and paste any of the EBNF shown bellow at https://www.bottlecaps.de/rr/ui on the Edit Grammar tab then click on the View Diagram tab.

// SyntaxUAIBN.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

NUMBER ::=  float | integer
LISTE ::=  NUMBER LISTE?
UAIBN ::=  "BAYES" LISTE

//
// tokens
//

eol ::= "\n"

// SyntaxUAIMN.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

NUMBER ::=  float | integer
LISTE ::=  NUMBER LISTE?
UAIMN ::=  "MARKOV" LISTE

//
// tokens
//

eol ::= "\n"

// SyntaxFormula.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

EVAL ::=  expression expression*
expression ::=  integer | float | scifloat | operator | "(" | ")" | ident | ","

//
// tokens
//

eol ::= "\n"

// SyntaxO3PRM.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

O3PRM_UNIT ::=  IMPORT_UNIT? UNIT UNIT*
IMPORT_UNIT ::=  IMPORT_BODY IMPORT_BODY*
UNIT ::=  TYPE_UNIT | INTERFACE_UNIT | CLASS_UNIT | SYSTEM_UNIT
TYPE_UNIT ::=  TYPE_DECLARATION | DEPRECATED_INT_DECLARATION | DEPRECATED_REAL_DECLARATION
INTERFACE_UNIT ::=  INTERFACE_DECLARATION
CLASS_UNIT ::=  CLASS_DECLARATION
SYSTEM_UNIT ::=  SYSTEM_DECLARATION
CLASS_DECLARATION ::=  CLASS PREFIXED_LABEL ( extends CHAIN )? ( implements IDENTIFIER_LIST )? "{" CLASS_BODY* "}"
CLASS ::=  class
PREFIXED_LABEL ::=  label
CHAIN ::=  label ( dot label )*
IDENTIFIER_LIST ::=  IDENTIFIER ( comma IDENTIFIER )*
CLASS_BODY ::=  CLASS_PARAMETER | CLASS_ANON_TYPE_ATTR | CLASS_ELEMENT
CLASS_PARAMETER ::=  param ( int LABEL default INTEGER semicolon | real LABEL default FLOAT semicolon )
CLASS_ANON_TYPE_ATTR ::=  ( labels "(" TYPE_VALUE_LIST ")" LABEL_OR_INT | INT_TYPE_DECLARATION LABEL_OR_INT | REAL_TYPE_DECLARATION LABEL_OR_INT ) ATTRIBUTE
CLASS_ELEMENT ::=  CHAIN ( ARRAY_REFERENCE_SLOT | NAMED_CLASS_ELEMENT )
LABEL ::=  label
INTEGER ::=  integer
FLOAT ::=  float
TYPE_VALUE_LIST ::=  TYPE_LABEL comma TYPE_LABEL ( comma TYPE_LABEL )*
LABEL_OR_INT ::=  LABEL | INTEGER_AS_LABEL
INT_TYPE_DECLARATION ::=  int "(" INTEGER comma INTEGER ")"
REAL_TYPE_DECLARATION ::=  real "(" FLOAT_LIST ")"
ATTRIBUTE ::=  ( dependson IDENTIFIER_LIST )? "{" ( RAW_CPT | RULE_CPT ) "}" semicolon
ARRAY_REFERENCE_SLOT ::=  "[" "]" LABEL_OR_INT semicolon
NAMED_CLASS_ELEMENT ::=  LABEL_OR_INT ( REFERENCE_SLOT | ATTRIBUTE | AGGREGATE )
REFERENCE_SLOT ::=  semicolon
AGGREGATE ::=  "=" LABEL "(" AGGREGATE_PARENTS ( comma LABEL_LIST )? ")" semicolon
RAW_CPT ::=  "[" FORMULA_LIST "]"
RULE_CPT ::=  RULE RULE*
AGGREGATE_PARENTS ::=  IDENTIFIER | "[" IDENTIFIER_LIST "]"
LABEL_LIST ::=  LABEL_OR_INT ( comma LABEL_OR_INT )*
IDENTIFIER ::=  CAST? LINK ( dot CAST? LINK )*
FORMULA_LIST ::=  FORMULA ( comma FORMULA )*
RULE ::=  LABEL_OR_STAR_LIST colon FORMULA_LIST semicolon
LABEL_OR_STAR_LIST ::=  LABEL_OR_STAR ( comma LABEL_OR_STAR )*
INTERFACE_DECLARATION ::=  INTERFACE PREFIXED_LABEL ( extends CHAIN )? "{" INTERFACE_BODY* "}"
INTERFACE ::=  interface
INTERFACE_BODY ::=  CHAIN ( "[" "]" )? LABEL semicolon
TYPE_DECLARATION ::=  type PREFIXED_LABEL ( ( DISCRETE_TYPE_DECLARATION | EXTENDED_TYPE_DECLARATION ) | INT_TYPE_DECLARATION | REAL_TYPE_DECLARATION ) semicolon
DEPRECATED_INT_DECLARATION ::=  INT "(" INTEGER comma INTEGER ")" LABEL semicolon
DEPRECATED_REAL_DECLARATION ::=  REAL "(" FLOAT_LIST ")" LABEL semicolon
DISCRETE_TYPE_DECLARATION ::=  labels "(" TYPE_VALUE_LIST ")" | DEPRECATED_DISCRETE_TYPE_DECLERATION
EXTENDED_TYPE_DECLARATION ::=  extends CHAIN ( "(" MAP ")" | DEPRECATED_EXTENDED_TYPE_DECLARATION )
DEPRECATED_DISCRETE_TYPE_DECLERATION ::=  TYPE_VALUE_LIST
MAP ::=  TYPE_LABEL colon TYPE_LABEL comma TYPE_LABEL colon TYPE_LABEL ( comma TYPE_LABEL colon TYPE_LABEL )*
DEPRECATED_EXTENDED_TYPE_DECLARATION ::=  MAP
FLOAT_LIST ::=  FLOAT_OR_INT ( comma FLOAT_OR_INT )*
INT ::=  int
REAL ::=  real
FLOAT_OR_INT ::=  float | integer
TYPE_LABEL ::=  LABEL | INTEGER_AS_LABEL | FLOAT_AS_LABEL
INTEGER_AS_LABEL ::=  integer
FLOAT_AS_LABEL ::=  float
SYSTEM_DECLARATION ::=  system PREFIXED_LABEL "{" SYSTEM_BODY* "}"
SYSTEM_BODY ::=  label ( dot label ( ( dot label )* ARRAY? LABEL ( "(" PARAMETER_LIST ")" )? | ( "=" LABEL ARRAY? | inc LABEL ARRAY? ) ) | ARRAY ( LABEL ( "(" PARAMETER_LIST ")" )? | dot LABEL ( "=" LABEL ARRAY? | inc LABEL ARRAY? ) ) | LABEL ( "(" PARAMETER_LIST ")" )? ) semicolon
ARRAY ::=  "[" INTEGER "]"
PARAMETER_LIST ::=  PARAMETER ( comma PARAMETER )*
PARAMETER ::=  LABEL "=" ( INTEGER_AS_FLOAT | FLOAT )
INTEGER_AS_FLOAT ::=  integer
IMPORT_BODY ::=  IMPORT_DECLARATION
IMPORT_DECLARATION ::=  import CHAIN semicolon
CAST ::=  "(" LINK ( dot LINK )* ")"
LINK ::=  label | integer
LABEL_OR_STAR ::=  TYPE_LABEL | "*"
FORMULA ::=  string | float | integer

//
// tokens
//

eol ::= "\n"
dot ::= "."
comma ::= ","
colon ::= ":"
semicolon ::= ";"
import ::= "import"
type ::= "type"
class ::= "class"
interface ::= "interface"
extends ::= "extends"
system ::= "system"
dependson ::= "dependson"
default ::= "default"
implements ::= "implements"
param ::= "param"
labels ::= "labels"
int ::= "int"
real ::= "real"
inc ::= "+="

// SyntaxO3PRMR.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

o3prmr ::=  ( ( package Ident ";" )? ( import Ident ( as ( default | word ) )? ";" )* RequestBloc* | Command )
Ident ::=  word ( "." word )*
RequestBloc ::=  request word "{" Command* "}"
Command ::=  Observe | Unobserve | Query | SetEngine | SetGrdEngine
Observe ::=  IdentArray "=" ( word | float | integer ) ";"
Unobserve ::=  unobserve IdentArray ";"
Query ::=  query IdentArray ( and IdentArray )* ";"
SetEngine ::=  engine ( "SVED" | "SVE" | "GRD" ) ";"
SetGrdEngine ::=  grd_engine ( "VE" | "VEBB" | "lazy" ) ";"
IdentArray ::=  word ( "[" integer "]" )? ( "." word ( "[" integer "]" )? )*

//
// tokens
//

eol ::= "\n"
package ::= "package"
import ::= "import"
request ::= "request"
query ::= "?"
unobserve ::= "unobserve"
engine ::= "engine"
grd_engine ::= "grd_engine"
as ::= "as"
default ::= "default"
and ::= "&"

// SyntaxNet.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

STRING ::=  string
IDENT ::=  ident
ELT_LIST ::=  ( string | number | integer | ident )
PURE_LIST ::=  ELT_LIST ( ELT_LIST )*
LIST ::=  "(" PURE_LIST ")"
GARBAGE_ELT_LIST ::=  number | integer
GARBAGE_LISTS_SEQUENCE ::=  GARBAGE_NESTED_LIST GARBAGE_NESTED_LIST*
GARBAGE_NESTED_LIST ::=  GARBAGE_ELT_LIST | "(" GARBAGE_LISTS_SEQUENCE ")"
Net ::=  "net" "{" ( IDENT "=" ( IDENT | STRING | LIST )* ";" )* "}" NODE* POTENTIAL*
NODE ::=  "node" IDENT "{" ( IDENT "=" ( IDENT | STRING | LIST )* ";" )* "}"
POTENTIAL ::=  "potential" PARENTS_DEFINITION "{" RAW_DATA EXPERIENCE? "}"
PARENTS_DEFINITION ::=  "(" IDENT ( "|" ( PURE_LIST )? )? ")"
FLOAT ::=  number | integer
FLOAT_LIST ::=  FLOAT ( FLOAT )*
FLOAT_NESTED_LIST ::=  "(" ( FLOAT_NESTED_LIST FLOAT_NESTED_LIST* | FLOAT_LIST ) ")"
RAW_DATA ::=  "data" "=" FLOAT_NESTED_LIST ";"
EXPERIENCE ::=  "experience" "=" GARBAGE_NESTED_LIST ";"

//
// tokens
//

// SyntaxDat.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

MDPDAT ::=  VARIABLES_DECLARATION ACTION* REWARD_FUNCTION_GRAPH DISCOUNT TOLERANCE
VARIABLES_DECLARATION ::=  lpar "variables" VARIABLE* ")"
ACTION ::=  "action" ( IDENT | STRING ) FLOAT? TRANSITION_FUNCTION_GRAPH* COST_FUNCTION_GRAPH? "endaction"
REWARD_FUNCTION_GRAPH ::=  "reward" ( ( lpar SUB_FUNCTION_GRAPH ")" | lpar LEAF ")" ) | "[" OPERAND ( lpar SUB_FUNCTION_GRAPH ")" | lpar LEAF ")" ) ( ( lpar SUB_FUNCTION_GRAPH ")" | lpar LEAF ")" ) )* "]" )
DISCOUNT ::=  "discount" FLOAT
TOLERANCE ::=  "tolerance" FLOAT
VARIABLE ::=  lpar IDENT MODALITY_LIST ")"
IDENT ::=  ident
MODALITY_LIST ::=  IDENT_OR_INTEGER MODALITY_LIST?
IDENT_OR_INTEGER ::=  IDENT | integer
STRING ::=  string
FLOAT ::=  number | integer
TRANSITION_FUNCTION_GRAPH ::=  IDENT ( lpar SUB_TRANSITION_FUNCTION_GRAPH ")" | lpar TRANSITION_LEAF ")" )
COST_FUNCTION_GRAPH ::=  "cost" lpar SUB_FUNCTION_GRAPH ")"
SUB_TRANSITION_FUNCTION_GRAPH ::=  IDENT ( lpar IDENT_OR_INTEGER ( lpar SUB_TRANSITION_FUNCTION_GRAPH ")" | lpar TRANSITION_LEAF ")" ) ")" )*
TRANSITION_LEAF ::=  FLOAT ( FLOAT )*
SUB_FUNCTION_GRAPH ::=  IDENT ( lpar IDENT_OR_INTEGER ( lpar SUB_FUNCTION_GRAPH ")" | lpar LEAF ")" ) ")" )*
LEAF ::=  FLOAT ( FLOAT )*
OPERAND ::=  operand

//
// tokens
//

lpar ::= "("

// SyntaxBIF.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

BIF ::=  NETWORK ( VARIABLE | PROBA )*
NETWORK ::=  "network" ( IDENT | STRING ) "{" PROPERTY* "}"
VARIABLE ::=  "variable" IDENT "{" PROPERTY* LABELIZE_VAR PROPERTY* "}"
PROBA ::=  "probability" "(" IDENT ( "|" LISTE_PARENTS )? ")" "{" PROPERTY* ( RAW_PROBA ";" | FACTORIZED_PROBA ) "}"
IDENT ::=  ident
STRING ::=  string
PROPERTY ::=  "property" IDENT ( "=" STRING ";" | largestring | IDENT ";" | FLOAT ";" )
LABELIZE_VAR ::=  "type" "discrete" "[" NBR "]" "{" MODALITY_LIST "}" ";"
NBR ::=  integer
MODALITY_LIST ::=  IDENT_OR_INTEGER ( MODALITY_LIST )?
IDENT_OR_INTEGER ::=  IDENT | integer
LISTE_PARENTS ::=  IDENT ( "," IDENT )*
RAW_PROBA ::=  "table" LISTE_FLOAT
FACTORIZED_PROBA ::=  ASSIGNATION ASSIGNATION*
LISTE_FLOAT ::=  FLOAT ( ( "," | "|" )? LISTE_FLOAT )?
ASSIGNATION ::=  ( "(" LISTE_LABELS ")" LISTE_FLOAT | "default" LISTE_FLOAT ) ";"
LISTE_LABELS ::=  ( IDENT_OR_INTEGER | "*" ) ( LISTE_LABELS )?
FLOAT ::=  number | integer

//
// tokens
//

// SyntaxDSL.ebnf
// EBNF generated by CocoR parser generator to be viewed with https://www.bottlecaps.de/rr/ui
//

//
// productions
//

DSL ::=  "net" ( IDENT | STRING ) "{" HEADER_PART? CREATION_PART? NUM_SAMPLES? SCREEN_PART? WINDOWPOSITION_PART? BK_COLOR? USER_PROPERTIES_PART? DOCUMENTATION_PART? SHOW_AS? NODE* OBSERVATION_COST_PART? "}" ";"
IDENT ::=  ident
STRING ::=  string
HEADER_PART ::=  "HEADER" BLOC_PART
CREATION_PART ::=  "CREATION" BLOC_PART
NUM_SAMPLES ::=  "NUMSAMPLES" integer ";"
SCREEN_PART ::=  "SCREEN" BLOC_PART
WINDOWPOSITION_PART ::=  "WINDOWPOSITION" BLOC_PART
BK_COLOR ::=  "BKCOLOR" integer ";"
USER_PROPERTIES_PART ::=  "USER_PROPERTIES" BLOC_PART
DOCUMENTATION_PART ::=  "DOCUMENTATION" BLOC_PART
SHOW_AS ::=  "SHOWAS" integer ";"
NODE ::=  "node" IDENT "{" "TYPE" ident ";" HEADER SCREEN_PART? USER_PROPERTIES_PART? DOCUMENTATION_PART? PARENTS VARIABLE_DEFINITION EXTRA_DEFINITION_PART? "}" ";"
OBSERVATION_COST_PART ::=  "OBSERVATION_COST" BLOC_PART
HEADER ::=  "HEADER" "{" "ID" ident ";" "NAME" string ";" "}" ";"
PARENTS ::=  "PARENTS" "(" PARENTS_LIST? ")" ";"
VARIABLE_DEFINITION ::=  "DEFINITION" "{" "NAMESTATES" "(" MODALITY_LIST ")" ";" PROBA "}" ";"
EXTRA_DEFINITION_PART ::=  "EXTRA_DEFINITION" BLOC_PART
BLOC_PART ::=  "{" ( ANY | BLOC_PART )* "}" ";"
PARENTS_LIST ::=  IDENT ( "," IDENT )*
MODALITY_LIST ::=  IDENT_OR_INTEGER ( "," MODALITY_LIST )?
PROBA ::=  "PROBABILITIES" "(" RAW_PROBA ")" ";"
IDENT_OR_INTEGER ::=  IDENT | integer
RAW_PROBA ::=  FLOAT_LIST
FLOAT_LIST ::=  FLOAT ( ( "," | "|" )? FLOAT )*
FLOAT ::=  number | integer

//
// tokens
//

[phwuil]

Hi again, and thanks for your time.

1- did you see the first comment about common.{h|cpp} that allows to remove from template many shared functions ? Is it a change that you could find useful ? (I have to say that repeating the work of generating templates and common files does not attract me much :-)

2- Using BIF parser, and parsing for instance the files

https://gitlab.com/agrumery/pgmrepository/-/tree/master/bif/Mildew.bif (2M),
linux : 801.6ms
windows : 13,637.1ms

https://gitlab.com/agrumery/pgmrepository/-/tree/master/bif/Diabetes.bif (3M),
linux: 1,025.8 ms
windows: 12,855.4 ms

The computations themselves are comparable in time (and results) for the two platforms. So this really is a problem during the parsing

3- You're right about the simplicity of some grammars. :-) In this project, we are not dealing especially with grammars but with probabilistic graphical models, probabilistic inference and statistical machine learning : It is convenient for us to have the same description for all the languages we have to deal with (especially since the simpler can/will evolve) ... furthermore, we already have specific parsers for CSV (specific code) and for XML (using tinyXML).

[mingodad]

I just compiled and executed a naked grammar/parser on Linux/Windows and got the results shown bellow (see attached the source files):
On Linux:

./run-readbif.sh
Time taken 0 seconds 49 milliseconds
0.05user 0.00system 0:00.05elapsed 100%CPU (0avgtext+0avgdata 1808maxresident)k
0inputs+0outputs (0major+114minor)pagefaults 0swaps
Time taken 0 seconds 32 milliseconds
0.03user 0.00system 0:00.03elapsed 100%CPU (0avgtext+0avgdata 1708maxresident)k
0inputs+0outputs (0major+113minor)pagefaults 0swaps

On Linux with WINE:

./run-readbif.sh
Time taken 0 seconds 83 milliseconds
0.08user 0.01system 0:01.00elapsed 9%CPU (0avgtext+0avgdata 7568maxresident)k
0inputs+0outputs (0major+1109minor)pagefaults 0swaps
Time taken 0 seconds 35 milliseconds
0.03user 0.01system 0:00.05elapsed 94%CPU (0avgtext+0avgdata 7584maxresident)k
0inputs+0outputs (0major+843minor)pagefaults 0swaps

On Windows10 running on VirtualBox:

$ readbif.exe Diabetes.bif
Time taken 0 seconds 62 milliseconds
$ readbif.exe Mildew.bif
Time taken 0 seconds 46 milliseconds

readbif.zip

[mingodad]

The templates/frames are there to allow any customization a user my need.

[phwuil]

Thanks for that !

There is a lot of memory allocation in the codes associated with the parsing, but there is also a lot in general in our codes and the difference in execution time between windows and linux is never (and by far) as important as for reading files.

So I would say it would be worthwhile to adapt our code to your new version ... However, re-doing the separation of template codes to common.h and common.cpp (which seems to me to be something any user might want) is not on the agenda :-) we lack resources for that. I hope to find the time to do that at some point.

Thanks for maintaining cocoR, the information and testing!