Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Public/private results depend on how cache file was persisted #66

Closed
john-kurkowski opened this issue Mar 9, 2015 · 7 comments
Closed

Public/private results depend on how cache file was persisted #66

john-kurkowski opened this issue Mar 9, 2015 · 7 comments
Assignees
@floer32
Labels
hold: a PR is pending a PR is pending that will address the issue (including if it is "X birds 1 stone")

Comments

@john-kurkowski
Copy link
Owner

Repro

  1. TLDExtract(include_psl_private_domains=True).update()
  2. TLDExtract(include_psl_private_domains=False).extract('foo.appspot.com')

Expected: foo appspot com
Actual: foo.appspot com

The opposite order is also broken.

Related to #64.
@john-kurkowski john-kurkowski mentioned this issue Mar 9, 2015
Merged
@john-kurkowski
Copy link
Owner Author

I think the cache should always contain both public/private, and then results are filtered at runtime.

@john-kurkowski john-kurkowski mentioned this issue Apr 1, 2015
Closed
@mauricioabreu
Copy link
Contributor

I agree with what you said above. Cache should contain everything. It is light and makes the codebase easier to handle.

@john-kurkowski john-kurkowski mentioned this issue Mar 7, 2016
Closed
@wumpus
Copy link

wumpus commented Oct 28, 2016

With the current situation, all users of tldextract in a given container/pyenv/whatever have to agree whether they want public or public+private. Imagine the fun if module A and module B disagree...

@john-kurkowski john-kurkowski mentioned this issue Nov 9, 2016
Closed
@a1Gupta
Copy link

a1Gupta commented Feb 3, 2017

Did anyone find solution for this ? @john-kurkowski Are you working on it ? I want to use both options for include_psl_private_domains in different modules.

@john-kurkowski
Copy link
Owner Author

Nope, I don't think anybody's worked on it.

@john-kurkowski john-kurkowski mentioned this issue Sep 25, 2017
Open
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Feb 21, 2018
@brycedrennan
Cache entire public suffix list. Select at runtime.
e509f7b 
Addresses john-kurkowski#66 but is not backwards compatible.
@brycedrennan brycedrennan mentioned this issue Feb 21, 2018
Closed
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Feb 21, 2018
@brycedrennan
Cache entire public suffix list. Select at runtime.
b0cbad4 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Feb 21, 2018
@brycedrennan
Cache entire public suffix list. Select at runtime.
d308b92 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Feb 21, 2018
@brycedrennan
Cache entire public suffix list. Select at runtime.
e31272a 
Addresses john-kurkowski#66 but is not backwards compatible.
@brycedrennan brycedrennan mentioned this issue Mar 25, 2018
Merged
@brycedrennan
Copy link
Collaborator

This is now addressed in #144

@floer32 floer32 self-assigned this Mar 4, 2019
@floer32 floer32 added the hold: a PR is pending a PR is pending that will address the issue (including if it is "X birds 1 stone") label Mar 4, 2019
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 4, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
f6973c6 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 7, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
1a72b7a 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 7, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
5fea054 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 19, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
4c29a8b 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 19, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
e63d2fc 
Addresses john-kurkowski#66 but is not backwards compatible.
brycedrennan added a commit to CircleUp/tldextract that referenced this issue Mar 19, 2019
@brycedrennan
feat: cache entire public suffix list. Select at runtime.
07260a9 
Addresses john-kurkowski#66 but is not backwards compatible.
@john-kurkowski john-kurkowski mentioned this issue Mar 14, 2020
Closed
@brycedrennan brycedrennan mentioned this issue Oct 8, 2020
Merged
john-kurkowski pushed a commit that referenced this issue Oct 10, 2020
@brycedrennan
feature: private tlds can be used at call-time (#207)
2eba0e5 
This is a second attempt at doing what was done in #144. Addresses #66.

- Add `include_psl_private_domains` to the `__call__` method.  This is now something you can choose on a per-call basis.  The object level argument now is only a default value for each call.
- The entire dataset from publicsuffix.org is saved to cache
- Ensure no weird cache issues happen when using with different `suffix_list_urls` by using different filenames per `suffix_list_urls`
- Use filelock to support multiprocessing and multithreading use cases
- Update bundled snapshot to be the raw publicsuffix data. Need to look at performance impact of this.
- Breaking change `cache_file` => `cache_dir`
@john-kurkowski
Copy link
Owner Author

Closed via #207.

bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Oct 21, 2020
@bmwiedemann
Update python-tldextract to version 3.0.0 / rev 8 via SR 843031
b3ace58 
https://build.opensuse.org/request/show/843031
by user mia + dimstar_suse
- Update to 3.0.0:
  This release fixes the long standing bug that public and private
  suffixes were generated separately and could not be switched at
  runtime, john-kurkowski/tldextract#66
  * Breaking Changes
    + Rename `cache_file` to `cache_dir` as it is no longer a
      single file but a directory
      (john-kurkowski/tldextract#207)
    + Rename CLI arg also, from `--cache_file` to `--cache_dir`
    + Remove Python 2.7 support
  * Features
    + Can pass `include_psl_private_domains` on call, not only on
      construction
    + Use filelocking to support multi-processing and
      multithreading environments
  * Bugfixes
    + Select public or private suffixes at runtime
      (https://github.com/john-kurkowski/tldextract/issu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@floer32 floer32

Labels
hold: a PR is pending a PR is pending that will address the issue (including if it is "X birds 1 stone")
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@john-kurkowski @brycedrennan @mauricioabreu @wumpus @floer32 @a1Gupta