Generalization of the "settings" concept

[lizmat]

I've started renaming files in the rakudo repo to use the new .rakumod extension. Apart from the modules living in lib, it struck me that the files that make up the setting (in src/core.c|d|e) don't really need to have an extension at all, as they are just concatenated into a single "script" if you will, to build the setting of a language version.

So I was thinking: maybe these files shouldn't have any extension at all, or one different from .rakumod. And that maybe the associated core_sources file, should live inside the directory containing the setting file, give that a special extension (e.g. .rakusetting) and generalize the building of a setting to just look for a file with that extension in a directory, use the list of files in there to concatenate, and build a setting file from that.

It's all a bit vague, I agree. But it feels to me that we should take this renaming opportunity to look further into how the setting mechanism can be used.

[vrurg]

Two related headaches:

• editors/IDEs. No extension is the worst case. A new extension is questionable.
• the order of files is important. A list with the right order is mandatory.

We don't change the list that often anyway. I would just stick to .rakumod as we're not going to change it anymore.

[MasterDuke17]

My understanding is that the individual files are combined (almost) only for performance reasons (i.e., it's much faster to compile 68k lines of code in one file than spread across multiple files). If that's correct it seems a step backwards to get rid of the extension, like we'd just be papering over a hopefully-not-permanent performance workaround.

[niner]

No, the files are combined because they do form one single computation unit with lots of interdepending parts. Look at all the forward declarations in the setting. They wouldn't work across compunit boundaries.

[lizmat]

So how about changing the extension of the setting files to .raku ? That would indicate that it's Raku code, but that it is not a self-contained module?

[vrurg]

.raku might be confusing as it is supposed to be a script name. The role of core files is that of a module, not a script. Thus, .rakumod fits better their purpose. Say, if I given one and only choice: .raku or no extension – I'd rather chose no extension.

[lizmat]

The role of core files is that of a module,

But that's exactly my point. They give the appearance of being so, but they are not: they're just elements of a giant source-file: the naming is purely to give you an idea what you can find inside of it. But in some cases, is just wrong (e.g. look at what is inside src/core.c/Env.pm6.

[niner]

I see a lot of discussion about possible solutions, but what is the actual problem we're trying to solve here?

[lizmat]

1. the core setting files have .pm6 extension and should be changed, my question is to "what"?
2. possibly related: an idea to generalize the concept of a bunch of source files being a single compilation unit, possibly using special extensions and a specification of the order in which the files should be merged (aka the tools/templates/6.c/core_sources file) living in the same directory.

[niner]

1. the core setting files have `.pm6` extension and should be changed, my question is to "what"?

The obvious candidate is `.rakumod`, i.e. do the same as we did before.

2. possibly related: an idea to generalize the concept of a bunch of source files being a single compilation unit, possibly using special extensions and a specification of the order in which the files should be merged (aka the `tools/templates/6.c/core_sources` file) living in the same directory.

But what's the problem such a generalization would solve?

[lizmat]

1. easier adding of more language versions
2. make it easier for people to create their own "settings"
3. other, bigger non-core projects might benefit from the same approach and would not have to re-invent the wheel

[vrurg]

1. What does it make so difficult to add a new core version? Besides, it's something no to be done more often than once in two years.
2. How an extension would make it easier to create a setting?
3. They could follow the step of rakudo. Besides, the only way to avoid re-inventing the wheel is to provide tools to produce it. I.e. if an authoring tool provide this functionality this would do much more to unification of the process.

2. possibly related: an idea to generalize the concept of a bunch of source files being a single compilation unit, possibly using special extensions and a specification of the order in which the files should be merged (aka the `tools/templates/6.c/core_sources` file) living in the same directory.

To my view, an authoring tool would be the answer to this too.

Besides, I don't see why the internal structure of the core must affect how 3rd party modules are built? If we're looking for standardization of some processes we issue a paper and/or tools as the standard implementation. If we're looking for simplifying 3rd party development then it's only about the tools we provide.

And yet, it's not always possible to foresee how people would do things and wether the approach the core devs choose is really the best for them. I could point at Perl's Dist::Zilla which is by far is the best authoring tool for the ecosystem. Anybody was able to predict it's appearance?

So, I propose to focus on what's good or bad for the core in first place.