urlwatch-2.1 new version migration fails

[slavkoja]

The urlwatch 2.x has changed their configuration and cache storing system, but the migration from 1.X fails with:

urlwatch --edit
No jobs file found

        Migrating cache: /home/user/.urlwatch/cache -> /home/user/.urlwatch/cache.db

Removing: 0141a464b24bbc03f2bbba2bf91fe94179572238
Removing: 35ecdde4fad96b22d3fd11cee70f2bb141296e7a
Removing: 6d501928989a55a0325b63f450309da105bc4be8
Removing: a6d62130ca1dae859c462d0b9325bee60e449a21
Removing: 7f8dc223248751c43f5fff47399ad808332bba88
Removing: cdc6c20b4ed496b8b185d37c7966f4e31a6dabc5
Removing: e382ca109a55e69d45da802b58a0d37d37dd5024
Removing: f0f6be08f546f22bbd3b22c9b1650ea426666ea5
Removing: cc3acca41c7ee65dadbd2693d969eb18729472b6
Removing: d952b8a505165d60824ccd73a34a15e7035a4b2e
Removing: b06d40c441961c272fc4bb13f9a0494d7c64be9e
Removing: 86c5f26c01db07316e61516f95626d246d7735c8
Removing: cce3f7a7af3039f374199a849fb7e24d306695d9
Removing: 4242a29b794b46739bcb7144e599182827e5b596
Removing: b3eaf7d9df6af694d067475dc5c3bffd845a5c42
Removing: 56a05b6aa9c51fc2e29bfed42bc33c72ca83aeb2
Removing: 85bf86ea5490220da6be0418c53348b6a2b8b110
Removing: 22c201209094bfa9b508182022933236b5039b3b
Removing: 1921dd46f0c55c2e45ff5f68a9cc483971bee7dd
Removing: 06f09294cf250d4ffa3ebf9371129f678eb3e173
Parsing failed:
======
[Errno 2] No such file or directory: '/usr/lib/python-exec/python3.4/share/urlwatch/examples/urls.yaml.example'
======

The file /home/slavko/.urlwatch/urls.yaml was NOT updated.
Your changes have been saved in /home/user/.urlwatch/urls.edit.yaml

It seems, that the urls.yaml.example file is installed into /usr/share/urlwatch/examples/urls.yaml.example, but the urlwatch expects it in the /usr/lib/python-exec/python3.4...:

Simple simlink /usr/lib/python-exec/python3.4/share/urlwatch -> /usr/share/urlwatch solves the problem...

It is reported by me on Funtoo bugs too, you can see it at https://bugs.funtoo.org/browse/FL-3128

[thp]

How is urlwatch installed? Is Funtoo using a non-standard installation layout for Python packages?

[jlec]

@thp We (Gentoo as a Funtoo upstream) are using a wrapper to enable parallel installation and selection of python ABIs. so you can set e.g. python3.4 to be pythonsystem-wide. For that matter we are installing all scripts into /usr/lib/PYTHON_ABI and have symlink of the scripts name to out wrapper in /usr/bin.

Like

$ ll /usr/bin/urlwatch
lrwxrwxrwx 1 justin justin 31 Feb 15 08:37 /usr/bin/urlwatch -> ../lib/python-exec/python-exec2
$ ll /usr/lib/python-exec/*
/usr/lib/python-exec/python3.4:
total 16K
-rwxr-xr-x 1 justin justin 14K Feb 15 08:37 urlwatch*

/usr/lib/python-exec/python3.5:
total 16K 
-rwxr-xr-x 1 justin justin 14K Feb 15 08:37 urlwatch*

So urlwatchprobably expects something relative to the patch of the script instead of some absolute location.

[thp]

Yes, that's the case. I wonder if there's an easy way to determine the location? This is how urlwatch does it at the moment:

# Check if we are installed in the system already
(prefix, bindir) = os.path.split(os.path.dirname(os.path.abspath(sys.argv[0])))

if bindir == 'bin':
    # Installed system-wide
    examples_dir = os.path.join(prefix, 'share', pkgname, 'examples')
else:
    # Assume we are not yet installed
    sys.path.insert(0, os.path.join(prefix, bindir, 'lib'))
    examples_dir = os.path.join(prefix, bindir, 'share', pkgname, 'examples')

[thp]

If you have a generic solution, I'd be happy to integrate that. Otherwise, maybe your packaging should patch this part in urlwatch so that prefix gets calculated to a proper value even for Gentoo's installation layout?

[jlec]

There are two solutions which come into my mind

• Make the location accessible via an env variable (least preferred)
• Test if dir is present in ../share/..., /usr/share/...., /usr/local/....

[jlec]

Perhaps it is even accessible through python or setuptools it self, as you are using the data_files argument in setup.py.

[ghost]

any progress?

[ghost]

i'm thinking it's gentoo specific bug.

[jlec]

i'm thinking it's gentoo specific bug.

Yes and no. Our way of installing doesn't work with the buildsystems way, which in it self isfar from optimal.

[ghost]

any progress?

[thp]

This is "somewhat" fixed by 7dc0fb8 in the sense that if the example files are not found, they will simply be ignored instead of failing.