Language grammer

[vasucp1207]

Initial language grammer proposal

[vasucp1207]

Notation	Meaning
"..."	Literal token that must appear exactly
[ ... ]	Optional (0 or 1 occurrence)
{ ... }	Zero or more repetitions
( ... )	Grouping
|	Choice

TODOS:

• postfix and suffix expressions
• struct methods
• global vars and constant
• arrays
• comptime expression
• expression statement
• generics

# Program
program        = { item }
item           = import_def | function | proc_def | struct_def | enum_def | extern_def
               | var_def | const_def

import_def      = "import" IDENT { "." IDENT } ";"

# Generics
type_params     = "[" type_param { "," type_param } [ "," ] "]"
type_param      = IDENT

# Functions and Procedures
function        = [ "pub" ] [ "inline" ] "func" IDENT [ type_params ] "(" [ params ] ")" result block "end"
proc_def        = [ "pub" ] [ "inline" ] "proc" IDENT [ type_params ] "(" [ params ] ")" block "end"
result          = "->" type
params          = param { "," param } [ "," ]
param           = IDENT ":" [ "const" ] type
block           = { statement }

# Foreign Function Interface
extern_def      = "extern" ( "func" IDENT "(" [ extern_params ] ")" "->" type
                | "proc" IDENT "(" [ extern_params ] ")" ) ";"
extern_params   = extern_param { "," extern_param } [ "," "..." ]
                | "..."
extern_param    = IDENT ":" type

# Structs and Enums
struct_def      = [ "pub" ] "type" IDENT [ type_params ] "=" "struct" [ struct_members ] "end"
struct_members  = struct_member { struct_member }
struct_member   = struct_field ","
                | method_def
struct_field    = [ "pub" ] IDENT ":" type

method_def      = [ "pub" ] [ "inline" ] "func" IDENT [ type_params ] "(" [ params ] ")" result block "end"
                | [ "pub" ] [ "inline" ] "proc" IDENT [ type_params ] "(" [ params ] ")" block "end"

enum_def       = [ "pub" ] "type" IDENT [ type_params ] "=" "enum" [ enum_variants ] "end"
enum_variants  = enum_variant { "," enum_variant } [ "," ]
enum_variant   = IDENT [ "(" type_list ")" ]

# Statements
statement       = var_stmt | local_static_var_stmt | assign_stmt | defer_stmt
                | unsafe_stmt | control_flow_stmt | return_stmt | expr_stmt

var_stmt        = "var" IDENT [ ":" type ] "=" expression ";"
assign_stmt     = place_expr ( "=" | compound_op ) expression ";"

var_def  = [ "pub" ] "var" IDENT [ ":" type ] "=" expression ";"

# like c, does not change value in simultanous calls.
local_static_var_stmt = "static" "var" IDENT [ ":" type ] "=" expression ";"

const_def       = [ "pub" ] "const" IDENT [ ":" type ] "=" expression ";"

place_expr      = "*" "(" expression ")"
                | "*" IDENT
                | IDENT { ( "." IDENT ) | ( "[" expression "]") }
compound_op     = "+=" | "-=" | "*=" | "/=" | "%="
                | "&=" | "|=" | "^=" | "<<=" | ">>="

defer_stmt      = "defer" block "end"

unsafe_stmt     = "unsafe" block "end"

control_flow_stmt = control_flow_expr        # if/match/while/for used as a statement — no ";", it ends on "end"

return_stmt     = return_expr ";"            # return is statement-only, never nested inside an expression

expr_stmt       = expression ";"

comptime_expr      = "comptime" block "end"
control_flow_expr  = if_expr | match_expr | while_expr | for_expr
if_expr            = "if" expression block { elif_clause } [ else_clause ] "end"
elif_clause        = "elif" expression block
else_clause        = "else" block
return_expr        = "return" [ expression ]
for_expr           = "for" IDENT "in" expression block "end"
while_expr         = "while" expression block "end"

# Match expression
match_expr      = "match" expression [ match_arms ] "end"
match_arms      = match_arm { match_arm } [ else_arm ]
match_arm       = "case" pattern block
else_arm        = "else" block

pattern         = literal_pattern | variant_pattern
literal_pattern = INTEGER | FLOAT | CHAR | STRING | BOOL
variant_pattern = IDENT [ "(" [ pattern_params ] ")" ]
pattern_params  = pattern { "," pattern } [ "," ]

expression      = orelse_expr
orelse_expr     = or_expr [ "orelse" expression ]
or_expr         = and_expr { "||" and_expr }
and_expr        = comparison { "&&" comparison }
comparison      = bitor_expr { ( "==" | "!=" | "<" | ">" | "<=" | ">=" ) bitor_expr }
bitor_expr      = bitxor_expr { "|" bitxor_expr }
bitxor_expr     = bitand_expr { "^" bitand_expr }
bitand_expr     = shift_expr { "&" shift_expr }
shift_expr      = range_expr { ( "<<" | ">>" ) range_expr }
range_expr      = additive [ ".." additive ]
additive        = multiplicative { ( "+" | "-" ) multiplicative }
multiplicative  = unary { ( "*" | "/" | "%" ) unary }
unary           = ( "-" | "!" | "~" | "&" | "*" | "try" ) unary | postfix
postfix         = primary { suffix }
suffix          = "." IDENT
                | "(" [ call_args ] ")"
                | "[" expression { "," expression } [ "," ] "]" # handles both foo[index] and foo[type]
                | "?"

primary         = INTEGER | FLOAT | CHAR | STRING | BOOL | IDENT
                | control_flow_expr
                | comptime_expr
                | array_literal
                | "(" expression ")"

array_literal = "[" [ array_elems ] "]"
array_elems = expression { "," expression } [ "," ]

call_args       = call_arg { "," call_arg } [ "," ]
call_arg        = expression

type            = [ "?" ] base_type [ "!" ]

base_type       = "i8" | "i16" | "i32" | "i64"
                | "u8" | "u16" | "u32" | "u64"
                | "usize" | "isize"
                | "f32" | "f64"
                | "bool" | "char" | "str"
                | "*" type
                | array_type
                | slice_type
                | named_type
                | func_type
                | proc_type

array_type = "[" ( INTEGER | "_" ) "]" type
slice_type = "[" "]" type
named_type = IDENT [ "[" type { "," type } [ "," ] "]" ]

type_list  = type { "," type } [ "," ]
func_type  = "func" "(" [ type_list ] ")" result
proc_type  = "proc" "(" [ type_list ] ")"

# Lexical elements
INTEGER         = dec_literal | hex_literal | oct_literal | bin_literal
FLOAT           = dec_literal "." dec_literal
CHAR            = "'" ( char_char | escape ) "'"
char_char       = any character except "'" or "\"
dec_literal     = digit { digit | "_" }
hex_literal     = "0x" { hex_digit | "_" }
oct_literal     = "0o" { oct_digit | "_" }
bin_literal     = "0b" { bin_digit | "_" }
hex_digit       = digit | "a" | ... | "f" | "A" | ... | "F"
oct_digit       = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7"
bin_digit       = "0" | "1"
STRING          = '"' { string_char | escape } '"'
escape          = "\\" ( "\\" | '"' | "n" | "t" | "r" | "0" )
BOOL            = "true" | "false"
letter          = "a" | ... | "z" | "A" | ... | "Z"
digit           = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
whitespace      = " " | "\t" | "\n" | "\r"
line_comment    = "//" { any_char_except_newline }

[Aliqyan-21]

@vasucp1207 yes I also fixed postfix and suffix expressions.

[Aliqyan-21]

@vasucp1207 global vars and consts will be same as locals only right? or what?

[Aliqyan-21]

@vasucp1207 I added the static and inline keywords

[vasucp1207]

@Aliqyan-21 put static_var_stmt in item, we can define globals also at the top

[Aliqyan-21]

@vasucp1207 Yes done, and also global and const var will be handled in this.

Looking something like this in lang:

const PI: f64 = 3.14159;

proc main()
    io.print("value of pi is {}\n", PI);
end

[vasucp1207]

@Aliqyan-21, what are the behivour of expr_stmt is it going to return a value when it will be the last statement in a block?

pub func add(a: i32, b: i32) -> i32
    var x = a
    var y = b
    x + y                       <------------------ return value?
end

[Aliqyan-21]

this will be too ambigous, let's use return keyword for returning the value, thus then we can check during compile time too if the type of return value is right or wrong, this is also good because we will be using '!' for error handling right, so then it will be returning a union of i32 and error, so yeah let's use "return" ?

[vasucp1207]

I think we can make the return for expr_stmt optional its a good syntactic sugar.

[Aliqyan-21]

IDK @vasucp1207 , this is another complexity to parse, let's just keep return, bedrock is not about syntactic sugar.

[skadewdl3]

I'm not a fan of the var mut syntax. Writing var conveys that the ident is a variable, so it should be able to change, which makes a mut on top of it confusing. I'd rather have something like let and let mut, or simply const and var.

[Aliqyan-21]

@skadewdl3 we don't have mut keyword in grammar, we just have var for all mutable variables, if u want to declare something immutable then u use const.

[skadewdl3]

Then the bedrock doc discussion needs to be updated because of uses something like that.

[skadewdl3]

And this had to be updated too

var_stmt = "var" [ "mut" ] IDENT [ ":" ["?"] type ] "=" expression

[Aliqyan-21]

@skadewdl3 yes that's a mistake, it will be removed.

[nogieman]

@vasucp1207 and @Aliqyan-21 why there is no inline functions here?

[Aliqyan-21]

great observation nogieman, surely it's needed.

[Aliqyan-21]

@vasucp1207 we need to add + and - for pointers too in grammar yeah? for pointer arithmetic, inside unsafe block we can allow it...?

[vasucp1207]

@Aliqyan-21, yes we can do pointer arith inside the unsafe block on raw pointers, but for that first we have to decide how we should annotate the raw pointers?

var arr = [1, 2, 3, 4, 5]
unsafe {
    var p = arr[0] as mut raw         <--------- raw mutable ptr
}

I suggest first we should work on memory model before jumping into the unsafe semantics, what do you think?

[skadewdl3]

Why are we even enforcing mutability in unsafe regions? Just var p = arr[0] as ptr would be better; but yeah we should do the memory model first.

[Aliqyan-21]

todo:

• generic syntax

[Aliqyan-21]

Do we need blocks in bedrock?

other languages have these as, they use braces for it {...}, for scoping.

We could have something like:

block
...
end

[skadewdl3]

Since we're going with lua-esque blocks, I don't like having explicit blocks. Unless we decide to have some meaning other than just scoping to blocks (for ex, block expressions or if blocks are associated with lifetimes of values and them being deallocated).

[Aliqyan-21]

yes right, @vasucp1207 was working on some region based memory model so I was thinking about these blocks

[skadewdl3]

@vasucp1207 can you show us how you'd like the memory model tie in to the syntax?

[vasucp1207]

@skadewdl3 you can look to the memory model discussion I put some example there about region blocks

[skadewdl3]

Let's have region as a keyword, instead of block. It tells the user explicitly what they're doing in terms of memory allocation.

Unless blocks adds something on top of regions, region keyword should be fine.

[vasucp1207]

@Aliqyan-21 do import defs ends with ';'?

[skadewdl3]

Why? Nothing else uses ;

[vasucp1207]

Yup this does seems confusing we have no use of semicolon anywhere other than import defs

[Aliqyan-21]

it does, all statements, externs, etc. ends with semicolons.

[Aliqyan-21]

it's everywhere in our examples in our doc too:

[Aliqyan-21]

it's their in grammar.

[vasucp1207]

Uhh, I missed this one completely, I think this is not there before

[Aliqyan-21]

yes right, I added it but forgot to announce it.

[vasucp1207]

@Aliqyan-21, we also need a slice type for passing arrays to functions or to return from a function.

[Aliqyan-21]

I was thinking the same thing.

let's add this in grammar:

base_type       = "i8" | "i16" | "i32" | "i64"
                | "u8" | "u16" | "u32" | "u64"
                | "usize" | "isize"
                | "f32" | "f64"
                | "bool" | "char" | "str"
                | "*" type
                | array_type
                | slice_type   # slice type will be another base_type?
                | named_type
                | func_type
                | proc_type

slice_type = "[" "]" type

good?

[skadewdl3]

Why do we need primitive types likke i8, i16, etc. as literals? Let's just generalize it to ident, which means both inbuilt and user-defined types. Others should be defined in terms of the ident, for ex, slice type becomes something like: ident[]

[vasucp1207]

These are all handled as idents during lexing, its only after parsing that they are converted into specific ast nodes representing primitive types.

[skadewdl3]

Yeah, I suggest we let those be idents in the grammar, not separate rules. The parser can decide what's a builtin type and what's not.

[Aliqyan-21]

yes we do that right now.

[Aliqyan-21]

for defer suppose we want multiple statements like:
in zig:

        defer {
            if (!bump) {
                self.line = line;
                self.col = col;
                self.pos = pos;
            }
        }

then we need some kind of block for defer instead of:

defer_stmt      = "defer" ( var_stmt | assign_stmt | control_flow_stmt | return_stmt | expr_stmt )

we would want:

defer_stmt      = "defer" ( statement ) "end"

[vasucp1207]

It should be like this,

defer_stmt = "defer" block "end"
block = { statement }

[vasucp1207]

We overlooked a very important caviat, we decided on the type keyword for defining types like struct/enums, etc because we could use it for defining any new types, but we did not included it in the grammar.

We wanted something like this too type would be able to do:

type real = f32;
type koo = func(i32, str) -> i32;
type vectring = []str; // idk about this?

but yeah, so I was thinking that struct and enums are ofc special kind of types so defined in blocked (struct ... end)

and for aliases we use ';' instead of end.

and this is grammar I am thinking:

type_def       = "type" IDENT [ type_params ] "=" ( "struct" [ struct_members ] "end"
                                                    | "enum" [ enum_variants ] "end"
                                                    | type ";" )

[vasucp1207]

@Aliqyan-21, can you give an example of defining struct as a type

[Aliqyan-21]

type Person = struct
  ...
end

[Aliqyan-21]

for generics:

type Person[T,U] = struct
...
end

[Aliqyan-21]

@vasucp1207 we add this in grammar yeah?

[vasucp1207]

We have optional in grammer but how we gonna represent it, should we add a null keyword for this or none would be a better one?

var a = null;
retrun null;
or
var a = none;
return none;

[Aliqyan-21]

it should be nil

[Aliqyan-21]

@vasucp1207 @skadewdl3

For strings we had a discussion and we all settled with the null terminates strings like this { ptr: *T, len: usize } right.

and for using in extern functions we could use str.cstr() to convert this to null terminated string copy.

discussed here

So should we lock this in?