travis: build and test webui as new build environment #97
Conversation
.travis.yml
Outdated
| sauce_connect: true | ||
| apt: | ||
| packages: | ||
| - libjansson-dev |
There was a problem hiding this comment.
The additional build environment for webui is build with postgresql database, sets BUILD_WEBUI to true, requires the sauce_connect addon and depends on libjannson-dev package
There was a problem hiding this comment.
Please add libjannson-dev to the general Debian build dependencies.
There was a problem hiding this comment.
Into the dependencies handed out over downloads.bareos.org ? Or in the general travis build dependencies ? If you meant the first, I'll ask tomorrow how to do so.
| sudo xargs --arg-file /tmp/build_depends apt-get -q --assume-yes install fakeroot | ||
| dpkg -l | ||
| sed -e "s/^.*:.*:\s//" -e "s/\s([^)]*)//g" -e "s/|/ /g" -e "s/ /\n/g" /tmp/dpkg-builddeps > /tmp/build_depends | ||
| sudo apt-get -q --assume-yes install fakeroot |
There was a problem hiding this comment.
Regex for scanning the build dependencies had to be modified as the pipe in the webui dependencies (apache2-dev | apache2-prefork-dev ) resulted in errors
| @@ -10,7 +10,7 @@ | |||
| # Modified to make a template file for a multi-binary package with separated | |||
| # build-arch and build-indep targets by Bill Allombert 2001 | |||
|
|
|||
| BAREOS_VERSION := $(shell dpkg-parsechangelog | egrep '^Version:' | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' ) | |||
| BAREOS_VERSION := $(shell sed -n 2p ../core/src/include/version.h | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}') | |||
There was a problem hiding this comment.
The version that was generated out of the debian/changelog file came from webui/packaging/obs/bareos-webui.changes. This file is no longer maintained so I get the bareos version out of another file.
| self.driver = webdriver.Remote( | ||
| desired_capabilities=self.desired_capabilities, | ||
| command_executor=sauce_url % (self.sauce_username, self.access_key) | ||
| ) |
There was a problem hiding this comment.
This opens the connection into saucelabs. The desired_capabilities can be extended to use more platforms than just chrome on os x.
| job_id = self.job_start_configured() | ||
| self.job_cancel(job_id) | ||
| self.logout() | ||
|
|
There was a problem hiding this comment.
I missed this in the git diff, I will add this back into the pull request.
| pass | ||
| else: | ||
| return chromedriverpath | ||
|
|
There was a problem hiding this comment.
This new method searches for the right path where the chromedriver is located. On os x, /usr/lib/ is reserved for system access only. /usr/local/lib is the right path.
| self.assertEqual([], self.verificationErrors) | ||
| else: | ||
| self.driver.quit() | ||
| self.assertEqual([], self.verificationErrors) |
There was a problem hiding this comment.
sys.exc_info() is pythons method to check if exceptions were thrown. If there were none we pass the test success to saucelabs by using the sauceclient module.
| SeleniumTest.username = 'citest' | ||
| SeleniumTest.password = 'citestpass' | ||
| SeleniumTest.sauce_username = os.environ.get('SAUCE_USERNAME') | ||
| SeleniumTest.access_key = os.environ.get('SAUCE_ACCESS_KEY') |
There was a problem hiding this comment.
Set the test variables different if we are currently in a travis build.
SAUCE_USERNAME and SAUCE_ACCESS_KEY need to be set in Travis' build settings so that they will not be exposed.
.travis.yml
Outdated
| sauce_connect: true | ||
| apt: | ||
| packages: | ||
| - libjansson-dev |
There was a problem hiding this comment.
Please add libjannson-dev to the general Debian build dependencies.
.travis/all
Outdated
| export BAREOS_CLIENT_NAME=$HOSTNAME-fd | ||
| echo "--------- testing webui over selenium -----------" | ||
| echo "configure add console name=citest password=citestpass profile=webui-admin" | bconsole | ||
| python /home/travis/build/thraizz/bareos/webui/tests/selenium/webui-selenium-test.py -v |
There was a problem hiding this comment.
Full path is dangerous. And this will not work after merge.
.travis/travis_before_install.sh
Outdated
| @@ -1,12 +1,20 @@ | |||
| #!/bin/bash | |||
|
|
|||
| sudo apt-get -qq update | |||
| sudo pip install selenium sauceclient | |||
There was a problem hiding this comment.
Are these not available as Deb packages?
There was a problem hiding this comment.
No, because they are python modules. The installation and binding handling is always handled by pip.
.travis/travis_before_script.sh
Outdated
| @@ -9,24 +9,40 @@ print_header() | |||
| } | |||
|
|
|||
| cd core | |||
| if [ "${COVERITY_SCAN}" ]; then | |||
| if [ "${COVERITY_SCAN}" ] | |||
| then | |||
There was a problem hiding this comment.
This changes was not required and does not look better.
|
I applied most of the changes and will correct the bareos-webui dependencies tomorrow. |
|
Well, at least https://packages.ubuntu.com/trusty/python-selenium is available as Deb package.
Am 19. Juli 2018 22:31:16 MESZ schrieb "A. Schueler" <notifications@github.com>:
…thraizz commented on this pull request.
> @@ -1,12 +1,20 @@
#!/bin/bash
sudo apt-get -qq update
+sudo pip install selenium sauceclient
No, because they are python modules. The installation and binding
handling is always handled by [pip](https://pypi.org/project/pip/).
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#97 (comment)
|
|
https://github.com/bareos/bareos/blob/master/core/debian/control and https://github.com/bareos/bareos/blob/master/core/debian/control.in
Am 19. Juli 2018 22:38:55 MESZ schrieb "A. Schueler" <notifications@github.com>:
…thraizz commented on this pull request.
> @@ -37,9 +37,16 @@ matrix:
- env: DB=mysql
- env: DB=sqlite3
- env: DB=postgresql COVERITY_SCAN=1
+ - env: DB=postgresql BUILD_WEBUI=true
+ addons:
+ sauce_connect: true
+ apt:
+ packages:
+ - libjansson-dev
Into the dependencies handed out over downloads.bareos.org ? Or in the
general travis build dependencies ? If you meant the first, I'll ask
tomorrow how to do so.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#97 (comment)
|
|
I will need to revert the installation process of selenium back to installation via pip, as the apt package does not contain selenium.webdriver components. We rely heavily on those, for example all expected conditions are imported out of those components. This is also documented here: EDIT: The build confirms this; https://travis-ci.org/thraizz/bareos/jobs/406182788#L5024 |
The readme currently contains the saucelabs badge two times. @joergsteffens, choose what looks better, @fbergkemper recommended to add it as a new sponsors section. Maybe we can put it into the |
README.md
Outdated
| @@ -1,6 +1,7 @@ | |||
| # <img src="https://raw.githubusercontent.com/bareos/bareos/master/webui/public/img/bareos.png" alt="Bareos" /> | |||
|
|
|||
| [](https://travis-ci.org/bareos/bareos/branches) | |||
|
|
|||
| [](https://travis-ci.org/bareos/bareos/branches)<br><a href="https://www.saucelabs.com/"> <img src="webui/public/img/saucelabs_white_badge.svg" alt="Saucelabs" width="90px"/></a> | |||
There was a problem hiding this comment.
Is this the recommend way to handle this? Not using img src, that refers to an icon provided by sauce Lab? Also, wouldn't it be better to link to our sauce Lab test results?
There was a problem hiding this comment.
It was only sent to me via email, but I've found it hosted by them. The link will point to our sauce lab results as soon as they go live, I've noted this on my todo list.
- Modify travis files to build the webui as new environment - Modify selenium test to test the webui over saucelabs if built over travis - Clean dead code in webui
|
Unfortunately, the selenium test do all fail: https://travis-ci.org/bareos/bareos/jobs/437528496 |
|
Python client can't use EDIT: line 475-493 in the log you posted contain some warnings about what I expected, |
- Fix an error where python was not able to decode body content due to an older version of urllib3.
|
I needed to specify to use urllib3 at version 1.22 as python requests dont like versions above (latest is 1.23) or below: |
|
This is a bit difficult to reproduce as my travis builds always fail without a paid saucelabs subscription, but I believe that the commit fixes the failures (and also just run |
|
Luckily your commits are tested by Travis / SauceLabs inside this Pull Request. It still fails, however, the error message has changed. Can you please take another look at it? |
|
This is due to security rules of travis, as stated here:
|
|
I see. So I merged it now to check the status after merge. |
|
The tests can be skipped if we add
|
|
Documentation for skipping tests: https://docs.python.org/2/library/unittest.html#skipping-tests-and-expected-failures |
|
One reason for the problems have been, that enviroment variables have not been set in our travis environment. It also needed the variable TRAVIS to be set. Now the sauce connect seams to work fine.
Can you please check on that? |
|
As The if(os.environ.get('BUILD_WEBUI') == 'true'):
sauce_username = os.environ.get('SAUCE_USERNAME')
access_key = os.environ.get('SAUCE_ACCESS_KEY')
if sauce_username and access_key:
SeleniumTest.sauce_username = sauce_username
SeleniumTest.access_key = access_key
else:
raise ValueError("SAUCE_USERNAME and SAUCE_ACCESS_KEY environment variables not set.") |
How it works as a diagram
Latest build in private travis
Example webui test in Saucelabs