New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cookies: use updated public suffix data #2553

Closed
bagder opened this Issue May 7, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@bagder
Member

bagder commented May 7, 2018

I did this

Use curl to work with cookies

I expected the following

Recent PSL updates to have an effect.

curl currently only works with libpsl's built-in PSL database and thus it will slowly go out of date over time unless libpsl is updated with a newer built-in database. Some operating systems (linux distros) now rather provide an "publicsuffix-list" package that updates the PSL data, so if curl would just use the correct libpsl API it could be using more up-to-date PSL information.

This libpsl API: psl_latest, is available since libpsl 0.16.0 (Dec 2016)

curl/libcurl version

All

operating system

All

Reported-by: Yaakov Selkowitz

PSL = Public Suffix List, see publicsuffix.org
libpsl = https://github.com/rockdaboot/libpsl

@bagder bagder added the HTTP label May 7, 2018

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder May 7, 2018

Member

psl_latest() allocates memory, loads the appropriate file contents into that dynamic memory and returns a pointer to a new "context" that needs psl_free() to get freed after use.

We thus need to cache the loaded results somewhere for some amount of time after use to make this bearable and not cause too much of a performance impact. We can't make it globally cached without also making it not thread safe.

Member

bagder commented May 7, 2018

psl_latest() allocates memory, loads the appropriate file contents into that dynamic memory and returns a pointer to a new "context" that needs psl_free() to get freed after use.

We thus need to cache the loaded results somewhere for some amount of time after use to make this bearable and not cause too much of a performance impact. We can't make it globally cached without also making it not thread safe.

@bagder bagder added the help wanted label May 11, 2018

@monnerat

This comment has been minimized.

Show comment
Hide comment
@monnerat

monnerat May 16, 2018

Collaborator

We can't make it globally cached without also making it not thread safe.

Does that mean we have to keep one PSL per handle ? Or would we introduce share interface options for it ?

Collaborator

monnerat commented May 16, 2018

We can't make it globally cached without also making it not thread safe.

Does that mean we have to keep one PSL per handle ? Or would we introduce share interface options for it ?

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder May 17, 2018

Member

Does that mean we have to keep one PSL per handle ?

Yes. But we can make the multi handle own it so that at least the use case with multiple transfers using the multi api will only use a single copy.

Or would we introduce share interface options for it ?

I think we should. I think it should be separate from the existing cookie sharing since the PSL is more likely to be the same for all transfers in an application.

Member

bagder commented May 17, 2018

Does that mean we have to keep one PSL per handle ?

Yes. But we can make the multi handle own it so that at least the use case with multiple transfers using the multi api will only use a single copy.

Or would we introduce share interface options for it ?

I think we should. I think it should be separate from the existing cookie sharing since the PSL is more likely to be the same for all transfers in an application.

@monnerat monnerat closed this in 8541d02 May 28, 2018

@lock lock bot locked as resolved and limited conversation to collaborators Aug 26, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.