Catkin plugin: Refactor build. #175

Merged
merged 1 commit into from Dec 19, 2015

Conversation

Projects
None yet
2 participants
Member

kyrofa commented Dec 18, 2015

This PR contains a number of small refactors:

  • Change env() to only contain variables for run-time, and document each of them.
  • Only obtain dependencies upon pull()-- for build() they should just be ready to go.
  • Extract dependency resolution into a standalone function that can be tested in isolation.
  • Change build() to be less clever and more simple/verbose/robust.
  • Add source-space keyword to get rid of custom build().
  • Get unit testing coverage of the Catkin plugin to 100%.
@@ -25,6 +25,9 @@
- catkin-packages:
(list of strings)
List of catkin packages to build.
+ - source-space:
@sergiusens

sergiusens Dec 18, 2015

Collaborator

Is source-space the word used upstream, if not, it feels weird. Maybe just workspace?

@kyrofa

kyrofa Dec 18, 2015

Member

Yeah, line 262. Workspace would be confusing since that's what source/source-subdir should be pointing to (the catkin workspace). Catkin has a "build space," a "devel space," an "install space," and a "source space." And a workspace 😛 .

snapcraft/plugins/catkin.py
+ of `source` and `source-subdir` keys) needs to point to a valid Catkin
+ workspace containing another subdirectory called the "source space." By
+ default, this is a directory named "src," but it can be remapped via
+ the `source-space` key. It's important that the source space is not the
@sergiusens

sergiusens Dec 18, 2015

Collaborator

just a pointer for change if the above schema is changed

snapcraft/plugins/catkin.py
schema['required'].append('catkin-packages')
return schema
def __init__(self, name, options):
- super().__init__(name, options)
+ """Sanity-check the plugin config regarding source code paths.
@sergiusens

sergiusens Dec 18, 2015

Collaborator

__init__ is much more than this. I am not entirely sure this is the best one liner docstring

@kyrofa

kyrofa Dec 18, 2015

Member

Is it though? Even the super class only sets a few variables in preparation for more work... doesn't it?

@sergiusens

sergiusens Dec 18, 2015

Collaborator

I have no quarrel with what it does, just the docstring, not sure you noticed, but __init__ above is also a link 😇

@kyrofa

kyrofa Dec 18, 2015

Member

Alright, I removed the docstring and put the comment back where it was.

snapcraft/plugins/catkin.py
+
+ Catkin packages can specify their system dependencies in their
+ package.xml. In order to support that, the Catkin packages are
+ interrogated for their dependencies here. Since "stage-packages" are
@sergiusens

sergiusens Dec 18, 2015

Collaborator

We might to check for consistency on use of "" versus `` and document it.

snapcraft/plugins/catkin.py
@@ -252,71 +229,55 @@ def build(self):
'{}', ';'
])
@sergiusens

sergiusens Dec 18, 2015

Collaborator

no incentive to change this as well 😉

Nothing that can't be done with os.walk and re or str().replace

# Don't clutter the real ROS workspace-- use the Snapcraft build
# directory
catkincmd.extend(['--directory', self.builddir])
+ # Account for a non-default source space by always specifying it
+ catkincmd.extend(['--source-space', os.path.join(
@sergiusens

sergiusens Dec 18, 2015

Collaborator

ah it indeed is called source-space ...

+ # This command must run in bash due to a bug in Catkin that causes it
+ # to explode if there are spaces in the cmake args (which there are).
+ # This has been fixed in Catkin Tools... perhaps we should be using
+ # that instead.
@sergiusens

sergiusens Dec 18, 2015

Collaborator

not sure what Catkin Tools are but I'm inclined to say yes

@kyrofa

kyrofa Dec 18, 2015

Member

Yeah, I'll be investigating that. For your reference, Catkin Tools.

snapcraft/plugins/catkin.py
@@ -340,40 +352,41 @@ def __init__(self, ros_distro, ros_package_path, rosdep_path,
self._rosdep_cache_path = os.path.join(self._rosdep_path, 'cache')
def setup(self):
+ # Make sure we can run multiple times without error, while leaving the
+ # capability to re-initialize, by making sure we clear the sources.
+ shutil.rmtree(self._rosdep_sources_path, ignore_errors=True)
@sergiusens

sergiusens Dec 18, 2015

Collaborator

What is the intention of ignore_errors here? If it is to clean after use, maybe contextlib is better here.

This will ignore the errors even if only some files failed to remove, I'll make it your call though.

@kyrofa

kyrofa Dec 18, 2015

Member

Ah, good point. I threw that in there to make sure it was okay if the files didn't exist, but I should just be catching that exception. Good catch. Fixing.

@kyrofa

kyrofa Dec 18, 2015

Member

Fixed.

snapcraft/tests/test_plugin_catkin.py
- mock.call().setup()])
+ def test_schema(self):
+ plugin = catkin.CatkinPlugin('test-part', self.properties)
+ schema = plugin.schema()
@sergiusens

sergiusens Dec 18, 2015

Collaborator
schema = catkin.CatkinPlugin.schema()

as schema is a classmethod

@kyrofa

kyrofa Dec 18, 2015

Member

Fixed, thanks!

snapcraft/tests/test_plugin_catkin.py
# sourcedir is expected to be the root of the Catkin workspace. Since
- # source_subdir was specified to be the same as the root, this should
+ # source_space was specified to be the same as the root, this should
# fail.
with self.assertRaises(RuntimeError) as raised:
plugin = catkin.CatkinPlugin('test-part', self.properties)
plugin.pull()
@sergiusens

sergiusens Dec 18, 2015

Collaborator

this call is never reached. Might want to check the test coverage for this file to spot others and remove dead test code 😉

@kyrofa

kyrofa Dec 18, 2015

Member

Oh this isn't here to DO anything. This test is saying "at least one of these lines had better throw an exception." I went back a forth a few times where I wanted the safety checks, so this makes the test less fragile to future changes.

@sergiusens

sergiusens Dec 18, 2015

Collaborator

You can make this a oneline then ;-)

catkin.CatkinPlugin('test-part', self.properties).pull()

@kyrofa

kyrofa Dec 19, 2015

Member

Ah, indeed! Fixed.

Collaborator

sergiusens commented Dec 18, 2015

💯 on this, looks really good in the readability domain!

Just had a couple nitpicks here and there.
Nice job on the 100% coverage.

Member

kyrofa commented Dec 18, 2015

Thanks for the good review @sergiusens! I've pushed a new version that should address your concerns.

Collaborator

sergiusens commented Dec 18, 2015

Looks good, just adding one more comment about try/except versus checking. Using if here might be better, no functionality is lost so I won't block (another minor one about the testing strategy)

you have my 👍 regardless

Member

kyrofa commented Dec 19, 2015

@sergiusens Fixed both. You're right, the if probably conveys my intentions better. I'm going to go ahead and merge this, then.

Catkin plugin: Refactor build.
This commit contains a number of small refactors:

- Change env() to only contain variables for run-time, and document
  each of them.
- Only obtain dependencies upon pull()-- for build() they should
  just be ready to go.
- Extract dependency resolution into a standalone function that
  can be tested in isolation.
- Change build() to be less clever and more simple/verbose/robust.
- Add `source-space` keyword to get rid of custom build().
- Get testing coverage to 100%.

Signed-off-by: Kyle Fazzari <kyle@canonical.com>

kyrofa added a commit that referenced this pull request Dec 19, 2015

@kyrofa kyrofa merged commit 59434ff into snapcore:master Dec 19, 2015

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.8%) to 88.725%
Details

@kyrofa kyrofa deleted the kyrofa:catkin_refactor_build branch Dec 19, 2015

smoser pushed a commit to smoser/snapcraft that referenced this pull request Sep 14, 2016

kalikiana pushed a commit to kalikiana/snapcraft that referenced this pull request Apr 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment