Gentoo Integration Tests #115

allenh1 · 2018-01-10T01:56:51Z

Opening for visibility. This is very, very rough.

Step one here (to me, at least) is to get docker to update a Gentoo image to be ready to emerge ros-lunar/ros_base, which is currently hard-coded into the script.

Not sure if this is working, as we are emerging ~300 packages after the USE flag changes, so I'll report back whenever that finishes (hopefully no webkit).

allenh1 · 2018-01-11T21:21:39Z

@tfoote This is the current testing framework. Do you think this will suffice?

allenh1@hunter-laptop ~/work/superflore
$ python3
Python 3.5.4 (default, Nov 11 2017, 15:58:51) 
[GCC 6.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from superflore.test_integration.gentoo.build_base import GentooBuilder
>>> bob = GentooBuilder()
>>> bob.add_target('indigo', 'catkin')
>>> bob.run()
>>>> testing gentoo package integrity
>>>> testing package 'ros-indigo/catkin'
>>>> Running container with command string 'bash -c 'emaint sync -r ros-overlay && emerge ros-indigo/catkin''...
>>>> Docker container exited.
>>>> 'ros-indigo/catkin': building
{'ros-indigo/catkin': 'building'}

tfoote · 2018-01-11T21:30:40Z

That looks reasonable. Where building means that it successfully builds? Then it will be trivial to create a nose test for a select set of packages.

allenh1 · 2018-01-11T21:37:16Z

allenh1@hunter-laptop ~/work/superflore
$ python3                        
Python 3.5.4 (default, Nov 11 2017, 15:58:51) 
[GCC 6.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from superflore.test_integration.gentoo.build_base import GentooBuilder
>>> bob = GentooBuilder()
>>> bob.add_target('indigo', 'fake_package')
>>> bob.add_target('indigo', 'catkin')
>>> bob.run()
>>>> testing gentoo package integrity
>>>> testing package 'ros-indigo/catkin'
>>>> Running container with command string 'bash -c 'emaint sync -r ros-overlay && emerge ros-indigo/catkin''...
>>>> Docker container exited.
>>>> 'ros-indigo/catkin': building
>>>> testing package 'ros-indigo/fake_package'
>>>> Running container with command string 'bash -c 'emaint sync -r ros-overlay && emerge ros-indigo/fake_package''...
!!!! 'ros-indigo/fake_package': failing
{'ros-indigo/catkin': 'building', 'ros-indigo/fake_package': 'failing'}

Newly fixed output.

allenh1 · 2018-01-11T21:42:14Z

That looks reasonable. Where building means that it successfully builds?

Yes.

Then it will be trivial to create a nose test for a select set of packages.

That's how I'm seeing it, at least. This tech has already proven itself invaluable.

I think there should be a follow-up PR with the actual test mechanism built into it. My current thought is that the list is saved in the branch we are testing on the overlay, gento-bot-[random-ascii], and then superflore will be run by travis (or other CI) to check the PR.

Something like this:

superflore-check-ebuilds -f [ebuild-file]

As an added advantage, this may be run locally.

superflore-check-ebuilds ros-indigo/desktop_full ros-kinetic/desktop_full ros-lunar/desktop_full

tfoote · 2018-01-11T21:48:04Z

superflore/docker.py

    def build(self):
+        if not self.dockerfile_directory:


This looks like the only place that the dockerfile is used so maybe it should be an argument for this method instead of a constructor argument?

Yep, that's a good point.

tfoote · 2018-01-11T21:48:28Z

superflore/docker.py

@@ -20,9 +20,10 @@


 class Docker(object):
-    def __init__(self, dockerfile, name):
+    def __init__(self, dockerfile=None, name=None):


I don't see self.name being used anywhere is it necessary here?

I guess I missed that. It certainly isn't used.

tfoote

Two small comments. And it would be good to have some simple tests for pull and build.

tfoote · 2018-01-11T23:08:12Z

superflore/docker.py

+
+    def build(self, dockerfile):
+        dockerfile_directory = os.path.dirname(dockerfile)
+        if not dockerfile_directory:


This check doesn't seem right at the moment. It should somehow verify that the Dockefile is a valid dockerfile not that the directory exists.

Good point. Fixed.

Sorry I meant that you should not just check that it's a directory but check that the Dockerfile itself exists. (aka check the file that is passed exists and then lookup the directory to pass on.) And even better would be to see if the Dockerfile parses/validates but that's extra.

tfoote · 2018-01-11T23:09:03Z

tests/test_docker.py

@@ -18,14 +18,12 @@

 class TestDocker(unittest.TestCase):
    def get_image(self):
-        docker_file = resource_filename('repoman_docker', 'Dockerfile')
-        dock = Docker(docker_file, 'gentoo_repoman')
+        dock = Docker()


This whole function seems outdated now.

I was keeping it for legacy reasons. ;)

Removed.

allenh1 · 2018-01-13T19:09:14Z

@tfoote new coverage report.

allenh1@hunter-laptop ~/work/superflore
$ coverage run --source=superflore -m nose && coverage report -m
.............................
----------------------------------------------------------------------
Ran 29 tests in 2.391s

OK
Name                                               Stmts   Miss  Cover   Missing
--------------------------------------------------------------------------------
superflore/CacheManager.py                            20     13    35%   23-24, 28-33, 37-41
superflore/TempfileManager.py                         34     12    65%   32-37, 48-56
superflore/__init__.py                                10      4    60%   5-8
superflore/docker.py                                  40     26    35%   24-27, 30-32, 35, 38, 41-47, 50, 53-69, 74
superflore/exceptions.py                              12      3    75%   18, 28, 33
superflore/generate_installers.py                     57     48    16%   32-86
superflore/generators/__init__.py                      0      0   100%
superflore/generators/bitbake/__init__.py              3      1    67%   3
superflore/generators/bitbake/gen_packages.py        115     92    20%   40-103, 110-166, 174-182, 188
superflore/generators/bitbake/ros_meta.py             27     18    33%   24-27, 30-35, 38-51, 54-55
superflore/generators/bitbake/run.py                 113     94    17%   39-44, 51-199
superflore/generators/bitbake/yocto_recipe.py        112     92    18%   43-64, 67, 70-74, 77-89, 92-96, 99-101, 104-105, 113-116, 125-177
superflore/generators/ebuild/__init__.py               3      1    67%   3
superflore/generators/ebuild/ebuild.py               143      7    95%   125-127, 131-136
superflore/generators/ebuild/gen_packages.py         151    125    17%   45-113, 118-150, 155-211, 216-233, 236, 239
superflore/generators/ebuild/metadata_xml.py          33      0   100%
superflore/generators/ebuild/overlay_instance.py      37     25    32%   27-32, 35-45, 48-64, 67-68
superflore/generators/ebuild/run.py                  157    137    13%   40-43, 47-52, 59-250
superflore/repo_instance.py                           49     34    31%   26-35, 38-46, 49-57, 63, 69, 75, 81, 84-91
superflore/rosdep_support.py                          34      2    94%   83-84
superflore/test_integration/__init__.py                0      0   100%
superflore/test_integration/gentoo/__init__.py         0      0   100%
superflore/test_integration/gentoo/build_base.py      27     27     0%   15-51
superflore/utils.py                                  118     41    65%   50, 54-59, 108, 114, 121-124, 126, 132, 138-139, 144, 148-149, 156-177
--------------------------------------------------------------------------------
TOTAL                                               1295    802    38%

EDIT: Ok, I'm actually not sure why it's not counting most of the docker.py file. Do you have any idea why that would be the case?

allenh1 · 2018-01-13T19:31:50Z

Figured it out. If you have a folder named test_thing and a test_thing.py file, the tests are not run. I'm going to see if this is an issue with the latest version of nose.

allenh1 · 2018-01-14T02:06:26Z

The issue here seems to be an authentication error, since one needs to login before they can pull. As such, I'm adding a login function to the Docker class, as well as looking into OAuth for CI login.

…obnoxious to do without user interaciton. There's a reason Gentoo doesn't have an automated installer yet.

…o mask it for now.

as relax the requiered dockerfile directory arg in the __init__ function.

…mpt to build the requested target.

tfoote · 2018-01-17T00:59:08Z

ubuntu is an official repository so it doesn't have an organization/username element to the image you should use ubuntu:xenial as xenial is the tag

…up trying to interpret it.

allenh1 · 2018-01-17T01:02:47Z

it doesn't have an organization/username element to the image you should use ubuntu:xenial as xenial is the tag

Yep, figured it was just me. Thanks! Working fine now.

allenh1 · 2018-01-17T01:22:44Z

@tfoote Travis is really slow atm, but I suspect this one will pass.

tfoote · 2018-01-17T01:26:26Z

superflore/test_integration/gentoo/build_base.py

+        for pkg in sorted(self.package_list.keys()):
+            self.container.add_bash_command('emaint sync -r ros-overlay')
+            self.container.add_bash_command('emerge %s' % pkg)
+            self.conainter.login()


spelling conainter

Fixed. That line didn't need to be there in the first place, actually.

tfoote · 2018-01-17T01:26:55Z

superflore/docker.py

+    def login(self):
+        # TODO(allenh1): add OAuth here, and fall back on user input
+        # if the OAuth doesn't exist (however one finds that).
+        user = getpass('Docker user:')


Does this fallback if in non-interactive shells?

Nope. No it does not. I'll use the environment variables I set on the CI if they are available, then do a os.isatty(0) to throw an exception if they aren't on a tty.

tfoote · 2018-01-17T01:28:46Z

superflore/test_integration/gentoo/build_base.py

@@ -0,0 +1,52 @@
+# Copyright 2018 Open Source Robotics Foundation, Inc.


This whole file is not used yet?

Not at the moment. My plan was to use it in a follow up PR (since we're many commits in at this point).

But I suppose I can start the superflore-check-ebuilds in this PR.

allenh1 · 2018-01-17T17:47:55Z

@tfoote Ok, added the superflore-check-ebuilds command. I've got some documenting to do.

…er to use extended permissions (such as SYS_PTRACE, which is needed for building glibc).

* superflore.docker.py: Add pull function, as well as relax the requiered dockerfile directory arg in the __init__ function. * Add a GentooBuilder class that will spawn a docker container and attempt to build the requested target. * Add a function to clear the bash commands. * Add run function to build_base.py. This function tests each of the desired packages in the spawned docker container. * Added output to run function. * Refactor docker class to reduce parameters. * Remove outdated code and fix build condition. * Remove unused function. * Add test docker build. * Add test pull and test run. * Fix typo in setup.py. * Add a login function to the docker class * Build the test Dockerfile to test the run function. * Add tag flag to pull function. * Add support for environment variable login and detection of a tty for login. * Add a superflore-check-ebuilds command. * Add support for the yaml file holding the packages to test. * Document the new script. * Add privileged flag to the run function to allow docker to allow docker to use extended permissions (such as SYS_PTRACE, which is needed for building glibc).

allenh1 added the enhancement label Jan 10, 2018

allenh1 self-assigned this Jan 10, 2018

allenh1 added the in review label Jan 11, 2018

allenh1 requested a review from tfoote January 11, 2018 21:42

tfoote reviewed Jan 11, 2018

View reviewed changes

allenh1 requested a review from tfoote January 11, 2018 22:00

tfoote reviewed Jan 11, 2018

View reviewed changes

allenh1 force-pushed the add-gentoo-integration-test branch from 1e16f41 to f3d32b3 Compare January 12, 2018 17:02

allenh1 added in progress in review and removed in review in progress labels Jan 13, 2018

allenh1 requested a review from tfoote January 13, 2018 19:08

allenh1 added in progress and removed in review labels Jan 13, 2018

allenh1 removed the request for review from tfoote January 13, 2018 19:36

allenh1 and others added 7 commits January 13, 2018 22:18

This gets us most of the way. We are updating Gentoo, which is quite …

3c9e935

…obnoxious to do without user interaciton. There's a reason Gentoo doesn't have an automated installer yet.

All good up to this point. GLibC is currently not working, will try t…

ca7b35e

…o mask it for now.

Alter Dockerfile to be fit for pushing.

8aa597c

Added a few basic deps that should be there.

1c63abf

Moved dockerfile to ros/ros-overlay repo.

5e59f7f

superflore.docker.py: Add pull function, as well

b45c608

as relax the requiered dockerfile directory arg in the __init__ function.

Add a GentooBuilder class that will spawn a docker container and atte…

4c656ca

…mpt to build the requested target.

The linter ordering just seems to be arbitrary at this point. I give …

dc449c9

…up trying to interpret it.

allenh1 added 2 commits January 16, 2018 20:20

Add tag flag to pull function.

329e1f8

Add test for Docker pull.

19dfdeb

allenh1 added in review and removed in progress labels Jan 17, 2018

allenh1 requested a review from tfoote January 17, 2018 01:22

tfoote reviewed Jan 17, 2018

View reviewed changes

allenh1 added 3 commits January 16, 2018 20:45

Remove login from run function.

f9d73ba

Add support for environment variable login and detection of a tty.

acb6db1

Remove unnecessary '\'.

4e190ee

allenh1 added in progress and removed in review labels Jan 17, 2018

allenh1 added 2 commits January 17, 2018 12:15

Add a superflore-check-ebuilds command.

cd4887d

Add support for the yaml file holding the packages to test.

76a524a

Document the new script.

54152cd

allenh1 added in review and removed in progress labels Jan 17, 2018

Add privileged flag to the run function to allow docker to allow dock…

a88aaf0

…er to use extended permissions (such as SYS_PTRACE, which is needed for building glibc).

allenh1 requested a review from tfoote January 17, 2018 18:12

tfoote approved these changes Jan 17, 2018

View reviewed changes

allenh1 merged commit 59d1412 into master Jan 17, 2018

allenh1 deleted the add-gentoo-integration-test branch January 17, 2018 19:45

This was referenced Jan 29, 2018

Testing #40

Closed

Gentoo Build CI #82

Closed

herb-kuta-lge removed the in review label Apr 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gentoo Integration Tests #115

Gentoo Integration Tests #115

allenh1 commented Jan 10, 2018

allenh1 commented Jan 11, 2018

tfoote commented Jan 11, 2018

allenh1 commented Jan 11, 2018

allenh1 commented Jan 11, 2018

tfoote Jan 11, 2018

allenh1 Jan 11, 2018

tfoote Jan 11, 2018

allenh1 Jan 11, 2018

tfoote left a comment

tfoote Jan 11, 2018

allenh1 Jan 12, 2018

tfoote Jan 12, 2018

tfoote Jan 11, 2018

allenh1 Jan 12, 2018

allenh1 commented Jan 13, 2018 •

edited

Loading

allenh1 commented Jan 13, 2018

allenh1 commented Jan 14, 2018

tfoote commented Jan 17, 2018

allenh1 commented Jan 17, 2018

allenh1 commented Jan 17, 2018

tfoote Jan 17, 2018

allenh1 Jan 17, 2018

tfoote Jan 17, 2018

allenh1 Jan 17, 2018

tfoote Jan 17, 2018

allenh1 Jan 17, 2018

allenh1 commented Jan 17, 2018

		@@ -0,0 +1,52 @@
		# Copyright 2018 Open Source Robotics Foundation, Inc.

Gentoo Integration Tests #115

Gentoo Integration Tests #115

Conversation

allenh1 commented Jan 10, 2018

allenh1 commented Jan 11, 2018

tfoote commented Jan 11, 2018

allenh1 commented Jan 11, 2018

allenh1 commented Jan 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tfoote left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

allenh1 commented Jan 13, 2018 • edited Loading

allenh1 commented Jan 13, 2018

allenh1 commented Jan 14, 2018

tfoote commented Jan 17, 2018

allenh1 commented Jan 17, 2018

allenh1 commented Jan 17, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

allenh1 commented Jan 17, 2018

allenh1 commented Jan 13, 2018 •

edited

Loading