cookies: use updated public suffix data

[bagder]

I did this

Use curl to work with cookies

I expected the following

Recent PSL updates to have an effect.

curl currently only works with libpsl's built-in PSL database and thus it will slowly go out of date over time unless libpsl is updated with a newer built-in database. Some operating systems (linux distros) now rather provide an "publicsuffix-list" package that updates the PSL data, so if curl would just use the correct libpsl API it could be using more up-to-date PSL information.

This libpsl API: psl_latest, is available since libpsl 0.16.0 (Dec 2016)

curl/libcurl version

All

operating system

All

Reported-by: Yaakov Selkowitz

PSL = Public Suffix List, see publicsuffix.org
libpsl = https://github.com/rockdaboot/libpsl

[bagder]

psl_latest() allocates memory, loads the appropriate file contents into that dynamic memory and returns a pointer to a new "context" that needs psl_free() to get freed after use.

We thus need to cache the loaded results somewhere for some amount of time after use to make this bearable and not cause too much of a performance impact. We can't make it globally cached without also making it not thread safe.

[monnerat]

We can't make it globally cached without also making it not thread safe.

Does that mean we have to keep one PSL per handle ? Or would we introduce share interface options for it ?

[bagder]

Does that mean we have to keep one PSL per handle ?

Yes. But we can make the multi handle own it so that at least the use case with multiple transfers using the multi api will only use a single copy.

Or would we introduce share interface options for it ?

I think we should. I think it should be separate from the existing cookie sharing since the PSL is more likely to be the same for all transfers in an application.