make install / apt-get leaves stale files? (was: cobbler version: throws exception)

[igalic]

when running cobbler console (on master), I'm getting the following exception:

igalic@steak ~ % sudo cobbler version
### ERROR ###
Unexpected remote error, check the server side logs for further info
<class 'yaml.parser.ParserError'>:expected '<document start>', but found '<scalar>'
in "<string>", line 2, column 1:
gitdate = Mon Jul 28 14:33:22 20 ...
^

The log says,

Tue Jul 29 11:03:34 2014 - INFO | REMOTE version; user(?)
Tue Jul 29 11:03:34 2014 - INFO | Exception occured: <class 'yaml.parser.ParserError'>
Tue Jul 29 11:03:34 2014 - INFO | Exception value: expected '<document start>', but found '<scalar>'
in "<string>", line 2, column 1:
gitdate = Mon Jul 28 14:33:22 20 ...
^
Tue Jul 29 11:03:34 2014 - INFO | Exception Info:
File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 2058, in _dispatch
@param str file_path kickstart snippet file path
File "/usr/lib/python2.7/dist-packages/cobbler/remote.py", line 1385, in extended_version
os.mkdir(udir, 0755)
File "/usr/lib/python2.7/dist-packages/cobbler/api.py", line 235, in version
else:
File "/usr/lib/python2.7/dist-packages/yaml/__init__.py", line 93, in safe_load
return load(stream, SafeLoader)
File "/usr/lib/python2.7/dist-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/lib/python2.7/dist-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 39, in get_single_node
if not self.check_event(StreamEndEvent):
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 174, in parse_document_start
self.peek_token().start_mark)

[jmaas]

works for me with todays sources:

[root@cobbler cobbler]# cobbler version
Cobbler 2.7.0
  source: 709cffd, Mon Jul 28 14:33:22 2014 +0200
  build time: Tue Jul 29 14:52:57 2014
[root@cobbler cobbler]# cat /etc/cobbler/version
[cobbler]
gitdate = Mon Jul 28 14:33:22 2014 +0200
gitstamp = 709cffd
builddate = Tue Jul 29 14:52:57 2014
version = 2.7.0
version_tuple = [2, 7, 0]

it's strange you see yaml errors; the cobbler version command has been reworked to use ConfigParser instead of PyYaml

[igalic]

this is super strange.

I'm building this "from source", ala

        $(clean-dirs)
        ( cd $(bld_dir) && git clone $(pkg_src) )
        $(in_pkg_bld_dir) make
        $(in_pkg_bld_dir) make install DESTDIR=$(dst_dir)/$(pkg)

and then package it with fpm:

        $(fpm) -n $(pkg) -v $(pkg_ver) --iteration $(pkg_rev) \
                --description $(pkg_desc) --url $(pkg_url) \
                $(pkg_dep) \
                $(shell find $(dst_dir)/$(pkg)/etc/$(pkg)  -type f | sed -r 's|$(dst_dir)/$(pkg)(.*)|--config-files \1|g' ) \
                -p $(package_dir)/NAME_FULLVERSION_ARCH.TYPE \
                -C $(dst_dir)/$(pkg) .
        touch $@

This does, indeed, result in an /etc/cobbler/version file:

[cobbler]
gitdate = Mon Jul 28 14:33:22 2014 +0200
gitstamp = 709cffd
builddate = Tue Jul 29 08:23:20 2014
version = 2.7.0
version_tuple = [2, 7, 0]

[igalic]

To give you the idea where this is going: I'm trying to fix a puppet cobbler module's "cobbler distro add" semantics, which expects that a cobbler distro add will result in a profile added, too, and subsequently removes that. I'm trying to skip the remove profile part based on the version, but I appear to be having a much deeper set of issues.

(e.g.:

% sudo /usr/bin/cobbler distro edit --name=ubuntu-14.04.1-x86_64 --os-version=trusty
exception on server: '--os-version for breed ubuntu must be one of precise, saucy, oneiric, lucid, quantal, raring, given was trusty'

[jmaas]

Hmz, is trusty in your distro_signatures.json ?
Maybe you have some stale files somewhere doing nasty things?

For now it doesn't look like this is caused by Cobbler.... btw, your Puppet module looks kinda neat.
I will give that a try at work.

[igalic]

Trusty is in the signatures, that's why I'm so weirded out by all this behaviour.

As I mentioned, the package is generated from a fresh git clone. That is after $workdir/cobbler and $workdir/destination are rm -rfed. Then we do a make install that result (from dst) is packaged.

I've put up a gist with a full file listing of the resulting package (dpkg -L cobbler | sort | cut -f2 | xargs md5sum) including checksums on gist.
Maybe someone here can see if anything is missing or left behind.

[igalic]

btw, all the files that are in /usr/lib/*py*/**/*cobbler* et al belong to the cobbler package, so they must've come from make install:

igalic@steak ~ % find /usr/lib/py*/**/*cobbler* /usr/lib/py*/**/*koan* \
  -type f -exec dpkg -S {} + | cut -d: -f1 | sort -u
cobbler

[jmaas]

Any progress on this mystery? :)

[jmaas]

If stuff doesn't work after the make install (before packaging), let me know and i will re-open.
You're on debian/ubuntu and i don't test stuff on those platforms nearly enough... so it might still be a cobbler problem. On the other hand, i see plenty of people using cobbler on debuntu.....

[igalic]

so, the problem turns out to be this line in /etc/init.d/cobblerd:

[ -x /usr/loca/bin/cobblerd ] || exit 0

This made me (and the system) assume that the restart worked out fine, while actually nothing happened at all. Fixing this (cruedely with perl -p -i -e 's@/usr/local/@/usr/@g' build/config/cobblerd) means that now cobbler version returns,

igalic@steak ~ % sudo cobbler version
Cobbler 2.7.0
  source: e537aea, Fri Aug 1 16:47:26 2014 +0200
  build time: Mon Aug  4 14:53:27 2014
igalic@steak ~ %