Please support accessing an offline version of the list

[joshtriplett]

Some other implementations of the public suffix list use (and share) an offline version of the list. For instance, libpsl, Python's publicsuffix, Perl's Domain::PublicSuffix, and Haskell's publicsuffixlist all use the same shared copy, packaged in Debian as publicsuffix.

Please consider supporting the use of a shared copy on the system. You might also support falling back to a compiled-in version (perhaps provided in a separate crate to avoid having to update this one too frequently). And then, for anyone who wants to download the list themselves and keep it updated, you can provide a URL as a constant, and let callers download that using whatever HTTP library they already use and provide its contents. (That would also avoid having to deal with caching policies within this library.)

This would also address issue #1.

[rushmorem]

This library has always supported accessing an offline version of the list as well as pulling the list from arbitrary URLs. See List::from_path and List::from_url methods. In fact, List::fetch simply calls from_url to download the list from the official URL, falling back to their Github repo if the site is down for some reason.

What we don't do is ship with any offline version. This is a deliberate decision. The public suffix list is a living document which receives updates frequently. Some people actually use it to check if a domain is valid. If they base their information from an ancient version of the list they are bound to get incorrect results. Users should be able to make informed decisions about caching strategies for their particular use case.

Please consider supporting the use of a shared copy on the system.

By using the aforementioned from_path method, one can already use a shared copy on their system by simply calling that method with the path to the list on their system.

Please note that #1 has since been fixed. Thank you for your interest in this library.

[joshtriplett]

Would you consider adding a feature flag that crates depending on this could disable, then, to disable the corresponding dependencies and the support for from_url and fetch?

[rushmorem]

@joshtriplett I like this idea. I'm working on it right now. Will ship shortly. Thanks!

[rushmorem]

@joshtriplett This is now possible in v1.0.3. Thanks!

[rushmorem]

@joshtriplett In case you are interested, the latest version, v1.1.0 has a List::from_reader method inspired by this issue. It means that if you use your own library to download the list you don't need to save it first. You can just pass the response object to that method.

[joshtriplett]

@rushmorem Thanks! I'd wondered about a from_data method, but from_reader seems even better.

[rushmorem]

@joshtriplett My pleasure :) I'm glad you like it.

[rushmorem]

@joshtriplett You may also be interested in the new psl crate that caches the list offline but also automatically downloads an updated one if the one you currently have is too old. By default, it downloads a new copy every week but you can change the duration to anything you want.