ConstRef keyword

[Wosi]

In FreePascal 2.6.0 the ConstRef keyword has been introduced.
It's a parameter modifier like const or var.
More information about it here: http://wiki.freepascal.org/FPC_New_Features_2.6.0#Constref_parameter_modifier

I extended DelphiAST to handle constref properly.
I'm not sure if this project aims to be compatible to FPC related language features. If not then just let me know and it will only be present in my own fork.

[RomanYankovsky]

@sglienke @vintagedave discuss :)

[NickRing]

I think this brings Issue 36 in to discussion also... ?

[vintagedave]

@RomanYankovsky Discuss, eh? :)

I can see why it's useful to have other dialects supported (although I wonder how many - what about Oxygene, for example?) But I'm cautious about adding too many things to the base / core code that are not applicable to Delphi. I'd like to see it done in an extensible way if possible - but, I'm not sure what that would be. Enums etc for example can only be declared in one place, so it can't work just via polymorphism.

[sglienke]

I would actually welcome an architectural overhaul (including typed nodes) first or we end up with a huge clusferf**k that nobody wants to maintain before extending into other dialects.

Adding a keyword that is only supported by one dialect directly into the parser without any checks imo is wrong because that could cause parsing errors on dialects that don't have it.

[Wosi]

@NickRing I'm with you. Supporting different dialects would be really nice. But I don't know the best practice to archieve this goal? Dependency injection? Compiler directives? A derived TPasSyntaxTreeBuilder for each dialect (e.g. TPasSyntaxTreeBuilderD7, TPasSyntaxTreeBuilderXE, TPasSyntaxTreeBuilderFPC2_6)?

.

[RomanYankovsky]

I will not merge this for now, sorry. First we need to find a way to not put all possible dialects into a single lexer and parser. So... maybe later.

[Wosi]

I totally understand this decision. I'd really like to help improving the dialoect management but to be honest, I still don't understand all the lexer code. For example: Why and how do the TmwBasePasLex.FuncXX methods work? When is FExID being set and when Result?
I don't know when I can find the time to understand the code completely and to work on a concept for dialect management. Has anybody already a concrete idea for dialect management?

[sglienke]

The FuncXX methods are an optimization for keywords. Each keyword has a hash value that is represented by the XX. Imo this can be done different and in a way more understandable yet fast way. But hey who volunteers in rewriting the lexer from scratch and making sure that everything still works.

[vintagedave]

The numbers represent a hash? Crikey.

It would be easy to volunteer, since it would be a simple rename
refactoring... if there were unit tests to ensure the output didn't
change. I've been thinking about this before and lack of tests is (to me)
one glaring problem with the lexer. Does anyone else feel adding them
would be a useful addition? Now, how to add them - or what to test and why
and where and how - is another matter...

On 26 August 2015 at 16:05, Stefan Glienke notifications@github.com wrote:

The FuncXX methods are an optimization for keywords. Each keyword has a
hash value that is represented by the XX. Imo this can be done different
and in a way more understandable yet fast way. But hey who volunteers in
rewriting the lexer from scratch and making sure that everything still
works.

—
Reply to this email directly or view it on GitHub
#119 (comment)
.

[sglienke]

The Problem is not the name of the methods but the entire architecture (imo). If you want to add other syntax you have to change the very low hard-coded details of the lexer. If you add a keyword you have to know what FuncXX handles it because of the hash algo (see TmwBasePasLex.KeyHash) which is not very flexible imo.

[RomanYankovsky]

Yes, if we want an extensible architecture, lexer has to be rewritten from scratch.

[quartexNOR]

The way the pasParse architecture stands right now it's going to be tricky to isolate dialects without essentially writing a new parser (or derived versions of the existing one). I for one wanted support for at least a subset of smart pascal, but supporting oxygene, smart pascal, fpc and delphi from the same lexer/parser is going to be tricky i think. Or at least disorderly.

Good old castalia is fast and efficient because it's largely written around delphi with that particular dialect in mind, with lookup tables and hashing of keywords hardcoded into the mix.

To support more dialects I guess one could implement a "core" setup, which recognizes the bare-bone structures of a pascal program (procedure, function, datatype result, begin/end etc.) and then derive dialects from that. Some things will be simple, like hashing keywords on-the-fly and attaching callback handlers for them. Other things, like the actual parsing rules - requires quite a bit of work.

It's not impossible, but it will be time-consuming with all the tiny details.
For instance, in smart pascal you dont need to declare property-fields or access getter/setters, you can use a shorthand version, like this:

Type
TMyClass = Class(TObject)
Public
Property MyProperty:Integer;
end;

This is very handy and the property will only be called "MyProperty", it wont have any other accessors except read/write through the public interface. But this is very different from both delphi, fpc and oxygene. So the property keyword and sub-parser would have to be unique for that dialect (which also supports ordinary property declarations).

It may be best to focus on delphi syntax for now and instead try to come up with a more dynamic class architecture for dealing with keywords, symbols and the likes.

Having said that, adding a normal {$IFDEF SUPPORT_FPC} switch in the code and dealing with the very few subtle differences ad-hoc could be worth while. Since FPC is 97% or so compatible with Delphi, that requires the least amount of work to add support for

[Wosi]

I close this now. The disuccsion about dialects and architceture should better take place in #36.