Beaver does not run under Python2.6 on Ubuntu 10.04

[ChrisLundquist]

lusis/chef-logstash#54

Reports that beaver does not run under Python2.6.
This implies limited support for RHEL family platforms which update more slowly than Debian family.

Those may be outside your target, if so feel free to close this issue.

@rafaelmagu may have more details as this info comes from him.

[rafaelmagu]

Jose mentioned in #logstash that I should create a PR for 10.04 support, so I take that as indication that 10.04 should still be supported. Further clarification is needed.

I will get the exact error message from my server tomorrow.

[josegonzalez]

The zmq dependency should be removed, and we should throw a nice warning to any buttmunch that expects it to work without libzmq. That would take care of several deployment headaches.

Are there any issues with the python code itself that is a blocked on python 2.6? I think the upcoming move to generators for tailing logs might be an issue, but feel free to correct me if I'm wrong.

In any case, it should be possible to get support via a virtualenv if the system python is broken, no?

[josegonzalez]

I've removed the zmq dependency in b29a005.

[rafaelmagu]

Full error message:

Traceback (most recent call last):
  File "/usr/local/bin/beaver", line 6, in <module>
    import beaver.config
  File "/usr/local/lib/python2.6/dist-packages/beaver/config.py", line 67
    configs = {x[0]: x[1] for x in self._config.items(filename)}
                            ^
SyntaxError: invalid syntax

[josegonzalez]

Whats the proper syntax for that?

[rafaelmagu]

Not sure. Not proficient in Python enough to tell. :)

[josegonzalez]

http://stackoverflow.com/a/1747827/1515875

Can you modify the code to do that and then see if it is valid?

 configs = {x[0]: x[1] for x in self._config.items(filename)}

becomes

 configs = dict((x[0]: x[1]) for x in self._config.items(filename))

[rafaelmagu]

Same deal:

Traceback (most recent call last):
  File "/usr/local/bin/beaver", line 6, in <module>
    import beaver.config
  File "/usr/local/lib/python2.6/dist-packages/beaver/config.py", line 67
    configs = dict((x[0]: x[1]) for x in self._config.items(filename))
                        ^
SyntaxError: invalid syntax

[josegonzalez]

err use a comma, sorry.

[josegonzalez]

 configs = dict((x[0], x[1]) for x in self._config.items(filename))

[rafaelmagu]

That's better. Different error now:

usage: beaver [-h] [-m {bind,connect}] [-p PATH] [-f FILES [FILES ...]]
              [-t {rabbitmq,redis,stdout,zmq,udp}] [-c CONFIG] [-d DEBUG]
beaver: error: argument -t/--transport: expected one argument

Inspecting the init script I see: BEAVER_CMD=' beaver -t -c /opt/logstash/beaver/etc/beaver.conf'

The template must not be generating properly by Chef.

[rafaelmagu]

Had my output set to amqp. It seems the recipe only understands amq or rabbitmq. Not sure if the former is meant to be that or if it was a typo by lusis.

[rafaelmagu]

Alright, got it to work with 'rabbitmq'. But it's probably a good idea to trap errors when log files don't exist. Getting the following for a file that is missing from /var/log (a valid assumption):

Traceback (most recent call last):
  File "/usr/local/bin/beaver", line 69, in <module>
    configfile = beaver.config.Config(args.config)
  File "/usr/local/lib/python2.6/dist-packages/beaver/config.py", line 48, in __init__
    self._data = self._parse()
  File "/usr/local/lib/python2.6/dist-packages/beaver/config.py", line 64, in _parse
    raise Exception('%s is not a valid file path' % filename)
Exception: /var/log/mywebapp/application.log is not a valid file path

[josegonzalez]

Should it just continuously poll for that file if it is the only configured one and it doesnt exist?

[rafaelmagu]

Probably. Set a few seconds of sleep between polls.

By having it not error out when the file isn't found, we can have Beaver installed in any order through Chef (or Puppet). Say mywebapp is deployed after beaver. In the current version, Beaver would just exit before mywebapp had a chance to touch that log file.

[rafaelmagu]

Does beaver only check for file existence at the start?

[josegonzalez]

Looks like it. I didn't write the config.py stuff, but it sounds like this is a different issue than 2.6 support. Can you confirm that this works under 2.6 if the file exists with that change above? If so, we can ship the 2.6 change, and then you can file a bug on the other one.

[ChrisLundquist]

Thank you all, great work.

@rafaelmagu did you want to re-pull request this now?
lusis/chef-logstash#54

[rafaelmagu]

@ChrisLundquist Done: lusis/chef-logstash#57