Add support for macros that take parameters

[wheals]

Vezzra recently brought this up, and it had been something I wanted to take a crack at anyway.

I'm not really fond of the @1@ syntax, but it does the job of avoiding conflicts with FOCS syntax itself, which %1% would not.

[Dilvish-fo]

I think this MACRO_INSERTION definition could use some improvement. For our most typical use case, where the macro is used by itself on a line, as with the Xeno example, this looks fine to me. But if you were using it in the middle of some other text, it seems it could be at the the least horribly inefficient, but I think unless I'm mistaken, actually problematic. Consider the basic text

Ok this is some entry with a first macro useage [[ANY_MACRO(param1)]].  Then on the same line we also have [[ANY_MACRO(param2)]] and maybe a little more text.

When it hits that first ANY_MACRO, isn't s2 going to be all of

param1)]].  Then on the same line we also have [[ANY_MACRO(param2

?

It seems to me that rather than simply "+~_n" that you'd also want to exclude ")" in addition to the newlines-- it's not like anyone really needs to use a parenthesis as part of their macro substitution key.

[Dilvish-fo]

In addition to my comment above, I do also want to make sure you get a "Bravo!" here-- it looks to me like this will be very helpful.

[wheals]

Would making it non-greedy also fix that first issue? I can't really speak to the efficiency of either approach, though...

[Dilvish-fo]

Well, the non-greedy fixes it, but I think is less efficient-- with each nongreedy character it will try to move forward to the next parsing component, which I believes involving pushing some state info onto some kind of stack, and then will have to backtrack, again and again until it gets to the parenthesis. I think just excluding both newlines and parentheses would be much more efficient.

[Dilvish-fo]

it's not like anyone really needs to use a parenthesis as part of their macro substitution key.

I said that after just reviewing the code, and was a little rushed, didn't think about it enough-- that flaw in that statement is that it wasn't actually referring to the macro substitution key, it was referring to the macro substitution value, and people very well could reasonably want a parenthesis in it, and they could also want to have a comma in it too.

However, thinking this through a bit more, trying to deal with more complicated values would get rather, er, complicated. Particularly if we wanted to try making these things nestable the way our normal macros are. If we did also provide for more complicated substitution values, it would be ideal if simpler values could still be handled with this simpler syntax. So, I think I will do a bit of testing with the functionality as it is now, and then this PR should probably be ok to merge.

Just for brainstorming what might hopefully be a followup extension to this:

Rather than using a single character for the separator and a single character for the terminator, perhaps we could use some rare/odd character combination for them both. To allow things to also stay simple for the many times when the more complex handling isn't necessary, the basic treatment would be like you already have it, values comma separated and terminated by a parenthesis, except that if the value starts with a special combo (perhaps '@#@') then it goes on until that individual value is terminated with some special combo terminator (perhaps '@%@' ). The parsing of that could probably also keep track of nesting, I think.

[Dilvish-fo]

Thinking it through more (and testing) it turns out that no change is needed to be able to have a substitution value include a right parenthesis-- it simply can't directly include it, but the value can itself contain a macro which in turn has the parenthesis.

I'm combining the parser changes into one commit, with the scripting change being a second commit.

This is very nice, thanks again, wheals.