Pragma syntax

[ozra]

Sorry for pestering with issues, I just haven't felt this fired up about a programming language for 16 years. So, the only things that annoy one, among all this greatness, itch a little in the fingers..

The pragma syntax is plain hideous. It's mainly because of the dots. In my programmer-eyes at least, the dots 'tie' to their nearest identifier-looking tokens, rather than appear as out-of-the-way pragma-block-delimiters. The otherwise so lovely and clutter free looking code, gets sucker punched by it.

If I may so humbly suggest with my new adopter eyes, if such a small addition as the following notation could not be a lot clearer? They both could co-exist without parsing ambiguity, right? Could it please be considered? It would be such a C++'ish wart that I so dearly want to escape.

I could make a stab at it, but I definitely have to get up to speed with Nim first.

proc isCounter(s: PSym): bool {{inline}} =
  s.kind in {skResult, skVar, skLet, skTemp} and 
  {sfGlobal, sfAddrTaken} * s.flags == {}

proc isCall(n: PNode): bool {{inline}} =
  n.kind in nkCallKinds and n[0].kind == nkSym

[reactormonk]

The corresponding lines are https://github.com/Araq/Nim/blob/devel/compiler/lexer.nim#L854-L863 and https://github.com/Araq/Nim/blob/devel/compiler/parser.nim#L784, feel free to tinker.

[rbehrends]

Nested braces can occur normally as the result of combinations of array constructors and/or sets. E.g.:

import tables
{{1}:{2}}.toTable.echo

While I am not particularly enthusiastic about the pragma delimiters myself, I don't see an easy solution that doesn't introduce ambiguities. Note that pragmas can occur in a number of places, not just after a procedure declaration.

[ozra]

@reactormonk - thanks for the pointer.
@rbehrends - good catch. I stumbled upon Nim just yesterday, and wrote my first program about an hour ago (replaced apache web server with a quick hack using jester, sites work fine, yippie! :) - so I haven't gotten to far in to the depth of the syntax yet, so when I wrote above I had no clue whether it occured. Now I know, thanks.

There must be some, more out-of-the-way delimiter, one can introduce that doesn't clash, no?

I totally get that introducing shitloads of keywords is out of the question, so in that event pragmas are here to stay, and well.. Avoiding warts before 1.0 would be nice..

I'll let it sink in while I learn the ropes...

[reactormonk]

@ozra we ran outta delimiters a while ago, that's why. Maybe |pragma|? ^^

[dom96]

As with most suggestions like these I am reminded of the times when me and @Araq discussed the possibility of implementing them.

Some time ago we were considering the possibility of changing the pragma syntax, one possibility was to use @ i.e.

proc foo() @async = ...

The only problem is that it doesn't work very well when you have lists of pragmas, or when you have params given to pragmas: {.push: cdecl.} or {.raises: [OSError].}.

If you can come up with a syntax which @Araq likes then I think there is a high possibility that he will be ok with introducing it. Nim code currently uses a lot of pragmas, and this will not change. It is therefore reasonable to have a syntax for pragmas which are more user friendly.

Personally, I dislike {{pragma}}.

[rbehrends]

Pragmas can basically occur in three positions: to annotate a proc declaration, to annotate a var/let/const declaration, and as a statement. Proc and var/let/const declarations should not give rise to any syntactic ambiguities for pretty much anything, and pragma statements could be prefixed with a keyword (which would also make them easier to identify). This means that we could (e.g.) just use normal braces. For example, using basic braces to illustrate:

proc foo(x: int): int {importc: "foobar", header: "foo.h"} =
  ...

template `?=`(x, y: untyped): untyped =
  let (success, `x` {inject}) = `y`
  success

pragma {deadCodeElim: on}

Note: braces don't quite work because of term rewriting macros/templates, but the above should illustrate the principle. A prefix character rather than a grouping character could also work by having it chained for multiple pragmas (and one could do away with it entirely in a statement context), e.g.:

proc foo(x: int): int /exportc("foobar") /header("foo.h") =
  ...

template `?=`(x, y: untyped): untyped =
  let (success, `x` /inject) = `y`
  success

pragma deadCodeElim(on)

[ozra]

@dom96 - thanks for some historical context, it makes it easier to understand the flow of the project
@rbehrends - thanks for the narrowing down, to come to that conclusion with my current limited knowledge of the lang would have taken a while ;)
I really like the idea of the pragmakeyword for the statement scenario.

• It makes it clearer in that context
• We're "all" used to it from the langs of the ancient

@dom96 - Yes the double-birdies was more of a 'out of the blue' example to contrast with the dot-brach delimiters. Agreed it's not the ultimate. Honestly though, I do like the pipes-style. Having some kind of squarish looking identifier kind of lends itself well to the (slightly more) hardware near notions of pragmas compared to 'regular code'.

Both the @pragma and /pragma is off the map imho. @ just don't fit the job imo (though I know other langs use it) , / kind of "disappeared" in the code.

Also, I see a big future for Nim (as you veterans obviously do too), so the language will obviously grow wider, so ofcourse some care must be taken to not lock away possibilities because of a naive choice for pragma declaration.

I'm happy to hear that this is an issue with mind share. I'll continue thinking about it while I'm getting into the language, and if anyone comes up with a good proposition and it's liked, I can code it into place.

[ozra]

To think about: From #2742 which I stumbled upon from my func issue, and by extension, from C / gcc: The de-facto double underscores for pragmas/attributes in gcc, etc, lends to the idea of just replacing the dots with underscores. (I borrow your example, @rbehrends). It might not seem like an improvement, but I do think it is clearer, the 'mental link' to gcc-pragmas is just side-effect ;)
(disclaimer, I still don't know enough syntax and semantics to know if this causes clashes - and don't mind the obvious errornous use of importc etc. it's just for lexical show.)

proc foo(x: int): int {_ importc: "foobar", header: "foo.h" _} = x + 47

proc bar(y: int): int {_ noSideEffects _} =
    doStuff(y)

template `?=`(x, y: untyped): untyped =
  let (success, `x` {_ inject _}) = `y`
  success

pragma  deadCodeElim: on    # with the pragma keyword, delimiters could be optional

Or, to separate it further, visually, from param parens roundness:

proc foo(x: int): int [_importc: "foobar", header: "foo.h"_] = x + 47

proc bar(y: int): int [_ noSideEffects _] =
    doStuff(y)

template `?=`(x, y: untyped): untyped =
  let (success, `x` [_inject_]) = `y`
  success

[Araq]

I like @rbehrends' suggestion, but keeping {. .} for pragma statements is good enough. These are rare.

[josephwecker]

What about

proc bar(y: int): int @noSideEffects = doStuff(y)

proc foo(x: int): int @{importc: "foobar", header: "foo.h", noSideEffects}

# or any other sigil like $, %

Of course, this is just scratching the surface of possible brackets -{ ... }- <[ ... ]> [( ... )] |< ... >| ... (or any of the above minus the last character)...

Personally I like the {. .} though- less visual clutter than just about any of the others (including possibly @). The only one that probably comes close IMO is -{ ... }-

I like that it's not just plain brackets or mustaches. Even if it would cause no technical ambiguity it would imply a semantic relationship that doesn't exist and make code that much more difficult to grok at a glance.

[Araq]

@josephwecker, @ozra You can use {. } instead of {. .} since forever but it's not common to use this style.

[dom96]

@josephwecker I like that suggestion. It looks nicer while remaining backwards compatible with the old syntax.

[reactormonk]

I'd prefer one syntax instead of two, because then we'll have a WTF to newbies - unless we deprecate the old and make nimfix change it.

[ozra]

@josephwecker - I like all those suggestions. Regarding the current syntax, on the contrary I found the dots "stutter" too much in my eyes, and "hook into" closest word, which is cluttery to me. I find all your examples to look more like out-of-the-way delimiters. Agreed on single braces, it could easily distract former C++ers like me in a tired moment. When it comes to the -{ foo }-: can't that cause problems with parsing? (prefix-op, set/table, ... abort!)? Also [()] -> (array <= set/table ...) ?

@Araq - alright, one less dot :) I tried it out like that, with a space after the dot, and that's an improvement to my eyes.

@dom96 @reactormonk - if @Araq & community do find a neat replacement, I do think depreciation and nimfix would be the path forward.

In the kind of code I'll be on, I imagine I'll be resorting to pragmas a lot, which of course is why I feel strongly about it. But, I mean, worst case, I'll just highlight it away...

Only to further widen the issue: is there anything hindering putting the pragma after the =? - That also helps decluttering imo; like so:

# signature is cleanly read
proc foo (a, b : int) : int  =
    {. noSideEffect } 
    a + b

[josephwecker]

@ozra - I wasn't really thinking too hard about the lexer- those are just the first several bracket types off the top of my head. To muddy the water further- here are more (since pragmas ~= directives ~= 'important' comments).

If someone wanted to try something fun- give Nim the ability to attach macros to "custom blocks"- defined as flexibly as custom operators (but with any limits appropriate due to the lexer / ambiguity etc.)- then define (for example):

# For super-pragmas
block `-<{( )}>-`(a: expr) = # ... construct normal pragma ...

# For length
block `(|  |)`(s: string) = s.len
block `(|  |)`(v: vector) = v.norm

s = "The string"
echo "Bits: ", (|s|) * 8

This isn't a serious proposal- just fun to think about.

[Araq]

@josephwecker's proposal is not backwards compatible. @{} is a valid expression in the current language and it's unary @ followed by the set constructor {}.

[0x1eef]

All of this seems subjective, because I think the current syntax is better than all of the alternatives proposed so far. The alternatives I've seen are hard to read, unusual(swap "." for "_"), or just harder to reason about the syntax(the @ variant especially).

[ozra]

Re-visiting this after some time with Nim, I actually think @dom96 suggestion looks best, alt. some other prefix, alt. something else - just something saner before 1.0 hits the streets.
Also, I think it would look cleaner if proc pragmas was declared after the = instead of before, but I guess that would be a problem if using say @...

[filcuc]

To be honest i like the current syntax.
However why not using the @dom96 proposed syntax?

proc foo(x: int, name: string): @async, @importc("temp"), @header("temp") =
  ...

[Varriount]

My greatest annoyance with the current pragma syntax comes not from the characters that have to be typed, but the placement. Too often pragmas cause neat procedure declarations to be split into multiline messes, which (to my eyes) interrupts the 'flow' when reading. I'd be much happier being able to put pragmas in a python-esque position, right before the procedure declaration. I've even thought of a way to unambiguously do such a thing, through use of [..] and a compiler plugin which rewrites the AST.
Example of the new syntax, modified from an entry in winlean.nim:

[.importc: "RemoveDirectoryA", dynlib: "kernel32", stdcall.]
proc removeDirectoryA*(lpPathName: cstring): int32

[rbehrends]

As far as placing procedure pragmas is concerned, the following works:

proc fls(x: cint): cint
  {.importc, header: "<strings.h>".}

echo fls(64)

[Varriount]

@rbehrends That works well for procedures without bodies, but for procedures with them, it causes visual obscurity. If you indent the pragma more the the left, it's fine, until you have to put arguments on more than one line, or have a complicated return type - then you get a big block of type you have to mentally separate. I prefer the visual separation that the python-esque way creates.
For example:

proc tryInsertID*(db: TDbConn, query: TSqlQuery, 
                           args: varargs[string, `$`]): int64
                           {.tags: [FWriteDb], raises: [].} = 
  ## executes the query (typically "INSERT") and returns the 
  ## generated ID for the row or -1 in case of an error. 
  ...

Or what if we need to add more pragmas:

proc tryInsertID*(db: TDbConn, query: TSqlQuery, 
                           args: varargs[string, `$`]): int64
                           {.tags: [FWriteDb], raises: []
                             effects: [] .} = 
  ## executes the query (typically "INSERT") and returns the 
  ## generated ID for the row or -1 in case of an error. 
  ...

There are a number of combinations you can use, but none of them provide as much separation as the stated alternative for as large a number of cases.
Pragmas play a big part in Nim. They are used for macro application, attribute marking, and many other things - a syntactic element with such important and varied uses deserves visual distinction.

[ozra]

@Varriount +1 on being able to place proc pragmas differently. Either first in body, or as in your example: immediately before definition.

[dom96]

@Varriount +1 👍

[rbehrends]

@ozra They can be placed first in the body. Well, they still have to be before the = sign, but otherwise, that's already possible. E.g.:

proc example(x: int,
             y: int,
             z: int): string
  {.exportc: "int3_tostring".} =
  result = "(" & $x & ", " & $y & ", " & $z & ")"

[ozra]

@rbehrends - yeah the = is enough to make it less readable imo. It should go tightly with the signature to make the function sig a clear unit - it's these small visual triggers that clarify what construct one sees and their scope.

[rbehrends]

@ozra Well, then it looks like everybody wants something different here, and I'm not sure those preferences – especially those that are mutually exclusive – can be satisfied at the same time.

[ozra]

I'm totally fine with before the proc too. So, either way.

[GooRoo]

Pragmas with dots look just fine with the right font (due to ligatures)

* It's FiraCode in the example above.

I would vote for keeping it as is.
The only thing I (perhaps) also might like from the discussion above is the placement of pragmas before the proc declaration.

[dom96]

The only thing I (perhaps) also might like from the discussion above is the placement of pragmas before the proc declaration.

That would be nice, but I feel it would need to extend to everything:

• Variable declarations
• Type definitions

etc.

[ozra]

@GooRoo - how do you visually distinguish those periods from middots!?? Sure - in that specific context, but...

[GooRoo]

@ozra I never use middle dots in my code :)
I don't think it's a problem as in all other places dots look like normal dots.

[Varriount]

@dom96 I like the idea of having pragmas above procedures. Perhaps the reserved [..] syntax cold be used?

[dom96]

That could work

[lightness1024]

so where are we with this ? as a beginner in Nim I've been surprised by the position of pragma in Nim. And it's already hard enough to remember : and =, having to put the pragma in BETWEEN those has got me pondering and refer to example again to be sure. python @ puts in above, C# puts it above, C++11 ([[]] annotations), and OpenMP pragmas are above too.
Let's take the {.thread.} pragma and {.nosideeffect.}, they are quite important, I would like it not be hidden all alone on the right.
check this out, at the end of the page:
https://nim-lang.org/araq/concurrency2.html
can you skim the code and guess quickly where are the thread procs and the nothread procs ?

[siliconvoodoo]

Especially, {.this: self.} seems to be supported above: https://nim-lang.org/docs/manual.html#overloading-resolution-automatic-self-insertions

[cyruseuros]

How about

+[async]
proc foo(x: int): string =
    ...

Visually familiar to rustaceans, removes the signature clutter, and feels like adding a "property" to a proc. Kinda reminiscent of org-mode options too. Would this cause any syntactic clashes?

[Araq]

Definitely, currently it's valid Nim syntax, unary plus followed by an array construction...

[FedericoCeratto]

Nesting the pragmas under the proc definition seems more natural to me. A single character (e.g. @) should be enough. This could support multiple pragmas per line and also multiple lines for better readability:

proc tryInsertID*(db: TDbConn, query: TSqlQuery): int64 =
  @pure, raises: [], effects: []
  @tags: [FWriteDb]
  @deprecated
  ## executes the query (typically "INSERT") and returns the 
  ## generated ID for the row or -1 in case of an error. 

[mfiano]

As a newbie to Nim, I also find the {..} syntax a bit jarring, as well as the placement in procs when dealing with long signatures that have to be multilined. It would be nice to have a dedicated section for optional pragma declarations before the body, similar to Common Lisp's declare, or more Pythonic before the definition. I think the {..} syntax is edging too close to symbol noise for me.

[liquidev]

@FedericoCeratto keep in mind that statement pragmas like .push and .pop are a thing, so your syntax would most definitely clash with that.

[jprochazk]

I'm also a newcomer to Nim, coming from C-like language background. The placement feels like it gets "lost" once you have more than a single line. I would support keeping the syntax, but changing the position to after the proc, var, type, etc statement:

proc tryInsertID*(db: TDbConn, query: TSqlQuery, 
                  args: varargs[string, `$`]): int64
                  {.tags: [FWriteDb], raises: [].} = ...

var
  a {.noInit.}: array[0..1023, char]

type
  MyFlag* {.size: sizeof(cint).} = enum
    A, B, C, D
  MyFlags = set[MyFlag]

becomes:

proc {.tags: [FWriteDb], raises: [].} 
  tryInsertID*(db: TDbConn, query: TSqlQuery, 
               args: varargs[string, `$`]): int64 = ...

var
  {.noInit.} 
  a: array[0..1023, char]

type
  {.size: sizeof(cint).} 
  MyFlag* = enum
    A, B, C, D
  MyFlags = set[MyFlag]

Would this introduce any other ambiguity?

[Araq]

Would this introduce any other ambiguity?

I am quite sure this wouldn't introduce an ambiguity. It's ugly for my taste, but technically it would work.

[Varriount]

@jprochazk I've argued before about using syntax similar to Python, at least for procedures and types. Although in that case, the position for pragmas attached to procedures would be before proc.

[Araq]

Reasons not to do anything:

• Invasive change now that 1.0 uses {. .} and books have been written containing this syntax.
• You can change your editor to render {. .} in a more aesthetically pleasing way.
• The "there must be only one way to program" purgers would send PRs correcting the code to use the new, better syntax all the while 1.0 doesn't support it at all. So the people who value stability are stuck with the old syntax anyway.
• The old syntax is not bad enough to warrant these disruptions.