RFE: optional one-shot/cached macro expansion

[pmatilai]

Some macros are really expensive to expand, notorious examples include python/perl/etc related macros that invoke the interpreter to fish out a configuration path or such. When these are expanded repeatedly, it really adds up.

In a spec one can use %global to achieve this, but that's not possible from a macro file which is how most such macros are distributed.

[voxik]

Is there any benefit in having macro files? Wouldn't be better to deprecate them instead?

[pmatilai]

Um. And use what instead? The spec file format? Whose parser is the one of the most compilicated and loathed piece of code in rpm and would be difficult to lift out of librpmbuild to put it mildly, and as a consequence slow down every rpm startup in the process?

In contrast, the macro file format is a nice, declarative format that can be always safely read without side-effects. It could be extended without sacrificing that.

[voxik]

First seeing spec file and then trying to move some macros into macros file, it was very confusing, I can tell you. From packager POV, the macro file is the same as spec file, but you don't write %global everywhere for some unknown reason. Also the difference between %import and %load (where the later was introduced on my request) is not really obvious.

Obviously, you know the internals, but honestly, that is just implementation detail to me.

[pmatilai]

Well yes, people occasionally get confused about that. The difference is a big one though, its like comparing a shell script with an ini file. Both have their places.

[mlschroe]

Crazy thought: we could allow to use %define/%global in the macro file, as those are illegal macro names. I.e. we could allow this:

%foo hello
%define bar world

(rpm internally somewhat rewrites the %foo to %define foo anyway)

Then we could also allow

%undefine foo

which is something I wanted to have since quite some time.

The downside is of course that people will start to try to use %if statements, because the macro file looks too much like a spec file.

[pmatilai]

There are other downsides too, such as arbitrary side-effects and code execution on load (as opposed to time of use).

[mlschroe]

Well, we don't need to allow %global.

Anyway, let's move a step back and talk about this issue. This is about one shot macros. Implementation wise this is not hard, it's more a question of syntax.

It would be nice to have the following options:

• literal: do not expand the macro when using it
• global: put macro in global namespace
• expand: expand macro right away
• once: cache expanded macro (this issue)

So %global is %define plus the global + expand flags.

Macro names can't start with a -, so we could make use normal options:

%global foo bar
%define -g -x foo bar

As next step can can discuss how we support this in the macro files ;)

[pmatilai]

Yeah, %define with options is what I'd really like (I think I even mentioned somewhere surrounding the pending %literal).

In macro files the name limitation could be abused for similar effect by adding special characters in the front of the name. AIUI rpm5 used "." to denote a read-only macro, for our purposes eg "=" could mean "literal" and so on. It's pretty ugly of course.

I suppose we could place the macro flags after the name, using a separator such as ':'
That would fit in the macro file syntax easily, eg:

%_pyver:o %(python3 --version)

Not sure its any prettier than special characters in front, though 😂 For a moment I thought about putting the flags inside [] after the name, but I think we want to reserve that syntax for arrays if we ever get around to implement them.

OTOH anything putting stuff after the name would need to deal with this first:

$ rpm --define "foo:bar 2" --eval "%foo"
warning: Macro %foo needs whitespace before body
warning: Macro %foo needs whitespace before body
:bar 2

[pmatilai]

FWIW, a simple and generic way to implement this on macro level is to redefine the macro as a global on first evaluation, eg:

%python3_sitearch %{global python3_sitearch %(RPM_BUILD_ROOT= %{__python3} -Esc "import sysconfig; print(sysconfig.get_path('platlib', vars={'platbase': '%{_prefix}', 'base': '%{_prefix}'}))")}%{python3_sitearch}

Optimally what would use %literal (or the equivalent) instead of %global but we don't have that yet...

[DemiMarie]

What about rewriting the macro in Lua and stashing the result in a Lua global variable?

[pmatilai]

It's possible in Lua of course, the above was to point out a way to do it without, because not everybody is content with Lua.