Drop Python 2, implement Python 3, replace virtualenv with native python 3 implementation, drop EPEL parameter

[waipeng]

Pull Request (PR) description

This PR attempts to fix tests that have been failing and blocking all PR from being merged.

To support python3, we have to introduce a few new variables to support
this creation of a py3 virtualenv, and also to set wsgi python-home 1
it. Just using activate_this.py didn't work.

In test, we also choose to use a separate virtualenv
virtenv3-puppetboard because `virtenv-puppetboard' will be created as
a py2 virtualenv by the other tests.

This Pull Request (PR) fixes the following issues

Fixes #287

[bastelfreak]

thanks for all the work! I already fixed all the linter fails in another PR: #288

I'm not sure what's the best way to bring both PRs together. Do you want to cherry-pick my commits into your branch? Otherwise I can cherry-pick your changes into mine as soon as the acceptance tests pass.

[waipeng]

Thanks for the review!

Currently I'm unable to get the CentOS tests passing. This is due to this fix installs puppetdb so that puppetboard can talk to it. Without a working puppetdb, puppetboard throws a 500 errors. You will notice most of the other tests don't test for connection, maybe this is the case.

On a dockerized CentOS I am unable to get systemd to work properly to start puppetdb properly, it errors out with

systemd[1]: New main PID <PID> does not belong to service, and PID file is not owned by root. Refusing.

There are two choices:

1. Continue trying fix CentOS and docker and systemd, or wait for someone to jump in and help
2. Change the test to not test for connection, but test for correctness in the generated configs (not as good)

Do you have any ideas?

[bastelfreak]

in the past we had strange issues with centos and systemd in docker. using an older version of the container fixed that. Can you use centos7-64{image=centos:7.6.1810}' instead of centos7-64 in the .travis.yml and check if the tests pass afterwards?

[bastelfreak]

crosses fingers for the centos tests

[waipeng]

thanks for all the work! I already fixed all the linter fails in another PR: #288

I noticed the modulesync change removed BEAKER_HYPERVISOR=docker so I'm going to try to see if that passes.

[bastelfreak]

you can probably let the linter fix the new violations with bundle exec rake lint_fix

[bastelfreak]

I noticed the modulesync change removed BEAKER_HYPERVISOR=docker so I'm going to try to see if that passes.

yes, that's now set by a gem we require, voxpupuli-acceptance: https://github.com/voxpupuli/voxpupuli-acceptance/blob/master/lib/voxpupuli/acceptance/spec_helper_acceptance.rb#L5

[ekohl]

I opened #293 to convert to voxpupuli-acceptance. Entirely untested, but that's probably a good base. If that works, I'd suggest to rebase on that. Also keep the PR focused and not include a full modulesync for easier review.

[waipeng]

looks like both ubuntu1604-64 and centos7-64{image=centos:7.6.1810} have problems starting services via systemd. Any ideas how to fix them?

[bastelfreak]

I will rebase this on our master

[bastelfreak]

crosses fingers

[bastelfreak]

Hi. i did a lot of debugging to figure out why the virtualenv command isnt found during the tests. My internet just died and I will continue tomorrow

[waipeng]

So I've debugged a few different docker images

• CentOS 7 - systemd 219 - Fail at starting puppetdb
• Ubuntu 1604 - systemd 229 - Fail at starting postgresql
• Ubuntu 1804 - systemd 237 - Pass
• Ubuntu 2004 - systemd 245 - Pass starting puppetdb, but fail other tests (needs work)

I think at this point in time we are debugging systemd/docker issues, not puppetboard/puppet-puppetboard issues.

I'm not sure what is the best way forward. Hoping someone with systemd and docker experience can help?

[ekohl]

I would start with (at least locally) testing with vagrant_libvirt. Real VMs at least allow you to exclude failures due to Docker. There's moby/moby#38749 which causes a lot of these issues. Sadly, it's pretty much the only CI we have. I have thought about using podman (which is supposed to be a lot more friendly to systemd inside container) but haven't gotten around to playing with that.

[bastelfreak]

I currently use the following command to test on centos7 with virtualbox/vagrant:

PUPPET_INSTALL_TYPE=agent BEAKER_destroy=no BEAKER_debug=true BEAKER_PUPPET_COLLECTION=puppet6 BEAKER_setfile=centos7-64{hypervisor=vagrant} BEAKER_TESTMODE=apply bundle exec rake beaker

this currently fails with:

       	Error: Could not find command 'virtualenv'

which should be virtualenv-3. I assume this is a bug in the puppet-python module. I'm currently debugging this and also considering, at least for testing, to switch to pyvenv instead. We already use that in two other modules with python3 on centos 7 so we know it's working.

[vchepkov]

virtualenv should be retired, since python3.6 it's no longer needed

[ekohl]

I would change the title of the PR for the changelog. It ended up also improving docs, add a parameter for daemon opts, use python::pyvenv.

[waipeng]

Thanks @bastelfreak, @ekohl and all for your help!

Can we have a version released for this please? We are keen to use it.

[bastelfreak]

i want to get #302 in before we do a release