logging error

[mcrotti]

This is a non blocking error that I can capture only if I import logging module.

manuel@Manuel-NG:~>python
Python 2.7.2 (default, Oct 4 2011, 14:55:10)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
...

import tldextract
import logging
logging.basicConfig()
one, two, three = tldextract.extract('forums.bbc.co.uk/nano/micro.html')

ERROR:/Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/tldextract.pyc:error reading TLD cache file /Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/.tld_set: [Errno 2] No such file or directory: '/Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/.tld_set'
WARNING:/Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/tldextract.pyc:unable to cache TLDs in file /Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/.tld_set: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg/tldextract/.tld_set'

touch'ed and chown'ed .tls_set with no success

chown'ed the folder, everything worked fine. Something to review in the install process, under OSX? (mine is Lion 10.7.3)

[john-kurkowski]

Not a logging error. It's a permissions error on the .tld_set cache, shown only when you turn on logging.

If tldextract can't read/write that file, it falls back to the included TLD snapshot. Either change permissions like you did, or specify an alternate .tld_set path. See the caching section of the readme.

This is also a reminder for me to review pkg_resources, which might be able to find a better resource location than inside the egg.

[calccalc]

Seems this problem has not been fixed? I have a number of machines that I'd like to deploy this package to, and I don't want to change the dir privileges on all those machines. Even if I use the "cache_file" parameter in the initialization, but it still complains the file not found error every time it's started on a new machine. Can't it just quietly download the file if the file doesn't exist?

[john-kurkowski]

You're right, if the file doesn't exist, that's not an error, so don't log it. Pushed to master.

Still open to suggestions on a better place to store .tld_set.

[john-kurkowski]

Nobody's clamoring for a better default place to store .tld_set so closing this.

[ibnesayeed]

Why not to set the default cache_file path in the temp directory? Here is how to do it in cross-platform manner.

[john-kurkowski]

It's supposed to be a cache, not a temporary file that can be deleted anytime.

[c24b]

+1 Up!

[c24b]

The only way for now to silent this WARNING is
logging.getLogger("tldextract").setLevel(logging.CRITICAL)
but it really should be a cache not tmp file that requires to change permission of the file were the packages is install so in prod in multiple places will be non very handy

[sharmaeshaan]

You need to do a sudo chown -R username:group directory to change the permissions/ownership of the /Library/Python/2.7/site-packages/tldextract-1.0-py2.7.egg folder to allow the program to read/write files.

[iMerica]

This caching feature is nice in theory, but its a bit much to expect a 3rd party python package to have write access to the file system. Anyone running this in a Docker container will see this error/running by default.

Please consider making caching disabled by default or support for caching capability using more appropriate tools like Memcache or Redis.

[john-kurkowski]

Good explanation!

Trying to understand better. Are people bothered by the verbosity? Or that this library is trying to write at all?

Remember, this is a warning and the library will continue with full operation and up-to-date definitions. I thought the warning brought attention that something is misconfigured, and you're relying on a live HTTP request on every application startup. If you're good with that, it's easy to silence. See this FAQ entry.

[lifenautjoe]

Good explanation!

Trying to understand better. Are people bothered by the verbosity? Or that this library is trying to write at all?

Remember, this is a warning and the library will continue with full operation and up-to-date definitions. I thought the warning brought attention that something is misconfigured, and you're relying on a live HTTP request on every application startup. If you're good with that, it's easy to silence. See this FAQ entry.

Its not a warning over here. The request fails entirely with

error
[Errno 13] Permission denied: '/usr/local/lib/python3.7/site-packages/tldextract/.suffix_cache'

Perhaps good to let people know where the file was downloaded and for them to copy it to their repository somewhere in the top parts of the README ?

[john-kurkowski]

Its not a warning over here.

That's a bug, #209. Try 3.0.2. That should restore the former behavior.