[IDEA] Filter to generate random numbers

[yaisog]

A basic building block of software is the generation of random numbers. Filter operators are basic building blocks for WikiText. There is no functionality in WikiText yet to generate a random number (except maybe hash the current datetime, but that is not truly random).

To limit this to a minimum functionality, a filter random[] could either generate a uniformly distributed random number between 0 and 1, or a filter random[<max>] could generate an integer random number between 0 and <max>. Personally, I like the second option better.
Both can be converted into each other with math filters. To generate random text (e.g. lowercase), use random[25]add[97] as a parameter to charcode[]. Endless possibilities.

[pmario]

I'd prefer a filter like random[min],[max] with min and max as multi operands. If they are omitted, defaults are: min=0 and max=1

The input should be treated as a seed. if no seed exists it should be a real random number created by our crypto library.

If seed exists it will be a "pseudo random" number, which creates repeatable results. eg: [[something]randtom[]] would be a pseudo-random with the string something as a seed.

So <$let seed={{{ [random[1][6] }}} result={{{ [<seed>random[1],[20]] }}}> would be a very basic random generator, with repeatable results.IMO that should fit our D&D community very well.

[yaisog]

The seed is a good idea. I'm not so much a fan of the min parameter. This is too easily achieved with just random[max-min]add[min]. But if there's a general concensus for it, that would be OK, too.

More interesting is whether you want to output to be integers or floats. I was initially thinking of integers, as floats with e.g. 6 decimals could be generated from integer numbers with random[1000000]divide[1000000]. On the other hand, from float to integer could be done with random[<max>]round[] of course, so that is also possible.

But if your defaults are 0 and 1, then the output would have to be a float to be useful? If it were implemented with integer output, my proposal for the default max value would be maxint. Then again, I don't think that the default values will be useful in a significant number of cases.

[AnthonyMuscio]

An application I was thinking of would need a random function but I am not sure how to use the proposed random operator,

Consider this example

• for each word in some text remove the first and last character
• If any letters remain jumble their order
• Add back the first and last character

So here the random operator needs to help jumble the letters in the word.

• I can see how generate N integers for the position of the letters or move
• Bui you can see it will quickly get a bit complex.

I raise this here just to share another class of the use of the random operator.

• Not that I can see a way to make it easier.

[pmario]

random[<max>]round[] IMO round should not be used see the Note at MDN https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random#getting_a_random_integer_between_two_values

I was sure, that the native js Math function would allow a user defined seed. But I was wrong. ... It's a library, that I had in mind: https://github.com/davidbau/seedrandom ...

I'm not sure, if we can convince Jeremy to include one more dependency into the core. Even if the minified version is only 1.5kByte ...

On the other hand, if we can reach a consent, that the core always ignores inputs with the random[] operator, it would be easy for a plugin to create a "drop in" replacement plugin, that would allow a seed as an input.

---

More interesting is whether you want to output to be integers or floats.

I think floats are pretty much useless since you cannot compare them with "match" or "equal". So IMO the random operator should always create integers --> including min and including max as described at: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/random#getting_a_random_integer_between_two_values_inclusive

[pmario]

I raise this here just to share another class of the use of the random operator.

IMO scrambling characters in a string has not much to do with a random-number generator. What you talk about is a scramble[] operator which may be a new Idea issue

[yaisog]

I am not sure how to use the proposed random operator

From the string with the remaining N letters (after removing first and last), remove the random[N-1]add[1] letter and add it as first letter to the new string. Then, take the random[N-2]add[1] letter from the remaining original string and add it as second letter to the new string. Then, take the random[N-3]add[1] letter ... and so on until all letters are removed from the original string and added to the new string.

The filters are based on a random[N] implementation that will return an integer between 0 and N (including both). If an integer between 1 and N (not including 0) were returned, then the first filter would be random[N], and the others according.

@pmario: A useful option (suffix?) would be whether to include 0 in the output range.

[Jermolene]

There has been quite a bit of prior discussion on this, but the only thread I could find is #3712.

It's worth noting that the stumbling block before has been that these random operators have significant pitfalls that can make them hard to use.

For example, imagine a [all[tiddlers]chooserandom[]] operator that picks a random entry from the input list. Naively, one might expect that the following code would display a random note from my collection:

<$list filter="[all[tiddlers]tag[Note]chooserandom[]]" template="$:/core/ui/ViewTemplate"/>

However, the problem lies in the fact that the filter will, by design, return a different tiddler each time it is evaluated. That means that each time the refresh cycle runs, the filter will be re-evaluated a different tiddler displayed. The user experience would be that typing a character into the search box would cause a different tiddler to be displayed.

You can try out something similar with the following fragment of wikitext. Paste it into a tiddler and then watch as the displayed value changes each time a character is typed in the search box:

<$list filter="[<now '[UTC]YYYY0MM0DD0hh0mm0ssXXX'>]"/>

One solution to the problem of displaying a random tiddler at startup would be to choose the tiddler to be displayed in the startup actions, storing its title in a state tiddler, and then transcluding it where required.

So, for those reasons, I have reservations about adding random operators along the lines suggested. The safest alternative would be to arrange things so that random number generation can only occur as part of action strings, for example by providing an action widget that generates random values in various ways.

[pmario]

... So, for those reasons, I have reservations about adding random operators along the lines suggested.

I knew, we did discuss that, but couldn't find it at GG ... So may be the best way to go will be a plugin anyway,

Since there are at least 2 different usecases coming up in this thread already. There may be different implementations needed.

So we can implement custom filters with plugins and prefix them properly. eg: wl-random or something similar for WikiLabs random and so on. So we don't create name-clashes.

[yaisog]

For example, imagine a [all[tiddlers]chooserandom[]] operator [...]

This is related, but not the same. I'm talking about pure random number generation, which is not available in TW. Maybe I don't mind if I get a different number every time I press a key, if I have set things up accordingly. Google Sheets also updates all random numbers with every change to any cell – I actually use this to quickly generate multiple maths sheets for the kids.

Of course it will not work (directly) for showing a random tiddler. But I could easily create a button (or some otherwise triggered action) that puts a number created with

<$let total={{{ [all[tiddlers]tag[Note]count[]] }}} theChosenOne={{{ [random[1],<total>] }}}>
<$action-setfield $tiddler=<state tiddler> $value=<<theChosenOne>> />
</$let>

into some state tiddler, and a template that uses

<$list filter="[all[tiddlers]tag[Note]sort[]nth{<state tiddler>}]" template="$:/core/ui/ViewTemplate"/>

to show the corresponding tiddler until some action changes the state / random number.

I am explicitly talking about a basic building block of RNG, not any advanced concepts building upon that.

Buuuuuut, I see where this is going, and I will (maybe) just make another plugin or something, shamelessly stealing all of Mario's good ideas from above... ;-)

[pmario]

Buuuuuut, I see where this is going, and I will (maybe) just make another plugin or something, shamelessly stealing all of Mario's good ideas from above... ;-)

No problem ... If they work for me I'll probably use them. If they don't I'll steal the improved ideas back ;)

[Jermolene]

Hi @yaisog I just used the chooserandom operator to make the example simpler to explain, but the same logic applies to a primitive operator that just returns random numbers.

So, we recognise the utility of random number generation, the question is whether the issue I have raised is a significant problem, and if so, what mitigations we can put in place.

One concern is that these infinite refresh loops would have a significant impact on energy usage and hence power consumption, which is obviously particularly important on mobile devices. The counter argument might be that these infinite loops would only be encountered during debugging by the people coding them, nd they'd be unlikely to be encountered in a finished application.

From an architectural point of view, these infinite refresh loops shouldn't be possible: the original intention was that all filters should be deterministic (ie they always return the same result given the same set of variables and the state of the tiddler store). In practice, we have of course introduced one or two non-deterministic macros but as things stand it is not at all easy to set up a refresh loop. Hence my concern about any changes that make it easier to set up these loops.

I think there might be a few ways that the problem could be mitigated. For example, a really brutal one would be to only enable the random operators within action widget execution, and to always return zero otherwise. A friendlier version of the same restriction would be to replace the operator with an action widget that generates random numbers.

We could also choose to ignore the issue. The argument might be that the consequences of making the mistake that I have outlined would be immediately obvious to the user and that they would be able to correct it. I'd still be concerned that figuring out what has gone wrong and how to fix it might be fairly demanding.

[kookma]

@Jermolene
What if we can have a tm-uniform-rand?

[Jermolene]

@Jermolene
What if we can have a tm-uniform-rand?

Widget messages don't have return values, so the best we could do would be to have parameters defining a tiddler/field/index into which the random number should be placed. The advantage of an action widget is that we could make the value of the random number available within the scope of the widget (as we do for the new tiddler title with the action create tiddler widget).

[kookma]

Widget messages don't have return values, so the best we could do would be to have parameters defining a tiddler/field/index into which the random number should be placed.

That would be fine! A message can have param and this allows to set seed, min, max, type, ...
Then the value can be set in a temp field/index, or as you said, available in the scope of widget. So, there will be no risk of using a message generates random.

[yaisog]

I made this into a plugin. If you ever want to more deeply explore the possible refresh issues, check it out here:
https://talk.tiddlywiki.org/t/filter-operator-to-generate-random-numbers/6017.
I did not encounter any weird behavior in my experimenting with it.

[CrossEye]

@AnthonyMuscio:

• If any letters remain jumble their order

This is traditionally called a "shuffle". It can easily be built atop a random number generator, but it's definitely a separate idea.

[AnthonyMuscio]

However often when people say random they often actualy mean shuffle. My guess is this is what is needed 80% of the time for most tiddlywiki users. Perhaps the sort operator could have a shuffle order. I raise it now because its related to or can be supported by the same macro or operator.

[yaisog]

With the availability of the plugins, I feel there is no need to fight to push this into the core anymore. At least for me. The plugin might not be particularly visible to those who do not look for it here or on TWTalk, but then again, such users will also not raise this issue again on GitHub.

I will thus close this issue to make way for more important ones.