Support regexp in Web.urlWhitelist

[traumschule]

We want to let the bot only paste titles of .onion addresses:

> config help supybot.plugins.Web.urlWhitelist
If set, bot will only fetch data from urls in the whitelist, i.e. starting with http://domain/optionalpath/. This will apply to all commands that retrieve data from user-supplied URLs, including fetch, headers, title, doctype.  (Current value: http://wcgqzqyfi7a6iu62.onion)
> title http://wcgqzqyfi7a6iu62.onion
Tor Research Home
> config supybot.plugins.Web.urlWhitelist *.onion
Error: This url is not on the whitelist
> config supybot.plugins.Web.urlWhitelist .onion
Error: This url is not on the whitelist

Please support regexp in whitelists and/or document how to use them if they are.
http://doc.supybot.aperio.fr/en/latest/search.html?q=urlWhitelist

[Mikaela]

I think you can use config plugins.web.urlWhitelist "\.onion(?:[:\/]|$)" (via https://stackoverflow.com/a/38332446), however I didn't try it.

Somewhat unrelatedly, can I ask how you are running Limnoria to have it access Tor, are you also affected by #1349 (or using torsocks to avoid #1349)?

[jlu5]

FWIW the config option takes a list of prefixes, not a regexp in its current state
https://github.com/ProgVal/Limnoria/blob/master/plugins/Web/config.py#L76-L80

[traumschule]

Thanks for calling in!

config plugins.web.urlWhitelist ".onion(?:[:/]|$)"
The operation succeeded.
title wcgqzqyfi7a6iu62.onion
Error: This url is not on the whitelist.

how are you running Limnoria to have it access Tor, are you also affected by #1349

I'm using torsocks or if that fails proxychains because usually applications can't be trusted to do socks right without leaks.

FWIW the config option takes a list of prefixes, not a regexp in its current state

That's what i figured too, it's not the right option for what we want. Played a bit with
config supybot.plugins.Web.nonSnarfingRegexp but to no avail yet.