Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport source API to lorax-composer #370

Closed
wants to merge 227 commits into from

Conversation

bcl
Copy link
Contributor

@bcl bcl commented Jun 4, 2018

No description provided.

bcl and others added 30 commits September 29, 2017 10:15
Created by command:

/usr/bin/tito tag --keep-version
This reduces the amount of code in livemedia-creator to the cmdline
parsing and calling of the installer functions. Moving them into other
modules will allow them to be used by other projects, like the
lorax-composer API server.
This requires pylint and python-pocketlint from EPEL7 to work.
runpylint.py copied from upstream lorax.
The lorax-composer program will launch a BDCS compatible API server
using Flask and Gevent. Currently this is a skeleton application with
only one active route (/api/v0/status).

The API code lives in ./src/pylorax/api/v0.py with related code in other
pylorax/api/* modules.
also submit coverage report to Coveralls.io if environment is
configured
when default value is list or dict the default arguments are
instantiated as objects at the time of definition. This is significant
(exposing visible semantics) when the object is mutable. There’s no way
of re-binding that default argument name in the function’s closure. When
function is executed multiple times with its default value the value
will change between executions, possibly leading to strange side effects.

For more information see:
http://satran.in/2012/01/12/python-dangerous-default-value-as-argument.html
- fix import of mock module
- account for io.StringIO differences in Python 2
It appears that with libgit2 v0.24.6 reverse causes it to list them
newest first. In 0.25.1 it lists them oldest first. On both versions
just using SortMode.TIME gives the desired result of oldest first.
And fix the import of recipes (needs to have full path passed in).
Includes adding a lock for access to the git repo from the API.
This handles the recipe workspace, which is a temporary location to
store recipes before committing them to git. It will be used by the
/recipes/ API.
Recipe should have its version bumped based on the version from the
previous commit, and not be bumped on the first commit. Fix the code and
the tests.
This will be useful for creating Recipe objects from json created dicts.
This handles json data when the mime type is "application/json" and toml
recipes when it is "text/x-toml"
Also fix use of workspace in /recipes/info (was using filename instead
of recipe name and logic for changed was backwards).
bcl added 26 commits April 4, 2018 12:02
Created by command:

/usr/bin/tito tag
This makes error handling consistent and easier than a couple layers of
fields to fetch.
Created by command:

/usr/bin/tito tag
Instead of enabling lorax-composer.service enable lorax-composer.socket
and it will start lorax-composer on first access to
/run/weldr/api.socket
This adds support for the optional blueprint section [customizations].

Use it like this:

[customizations]
hostname = yourhostnamehere

[[customizations.sshkey]]
user = root
key = root user key
[[customizations.user]] and [[customizations.group]]
Otherwise the user creation fails when anaconda sees there is already a
group with that name. Log a warning and continue on.

(cherry picked from commit a363aee)
It was using /tmp/ which can fill up quickly when building images.
Default to /var/tmp unless the user passes --tmp /tmp/
Crashing can sometimes leave directories in /var/tmp/lmc-* so clean
those up after run_creator is finished.
Overrides the [yum] proxy setting in the config file.
A crash can also leave temporary lmc-* files, remove them as well.
Created by command:

/usr/bin/tito tag
You can use '*' wildcards and '?' for single character matching.
This lives under /api/v0/projects/source/*

See the documentation for details
It was chopping off an extra directory level due to realpath removing
the trailing / from the paths when they are setup.
Also remove an unneeded makedirs from test_server.py
When adding a source failed it wasn't being removed from the dnf object.
This fixes that, and returns an error when setting up the source fails.
Also adds a test for it.
yum TumRepository.dump() function cannot be used as a .repo file Add a
new function to write this in the correct format, and limited to the
fields we use.

Add a test for the new function.

Fix /projects/source/info to return an error 400 if a nonexistant TOML
source is requested. If JSON is used the error is part of the standard
response.

Update test_server.py to check for the correct error code.
yum also has a cache it uses for listEnabled(), but the cache isn't
invalidated when a repo is deleted it any following metadata update
will fail because it is still using the deleted repo.

We are forced to use the heavy hammer on a yum private variable yet
again to force the cache to be cleared so that it won't crash.
This adds the sources command which can be used to list, add, change,
and delete sources using the TOML formatted source file.
Yum needs to have some other attrs setup on the YumRepository object, so
use the function provided to ensure that everything is correct. Also
switch the related functions to use a dict instead of a YumRepository
object.
The current version of libgit2 available (0.26.3) has different behavior
with SortMode.TIME. It works correctly when left at the default (which
is also how the rawhide version works).
@coveralls
Copy link

Pull Request Test Coverage Report for Build 243

  • 195 of 249 (78.31%) changed or added relevant lines in 7 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+2.08%) to 40.547%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/pylorax/api/queue.py 0 2 0.0%
src/pylorax/api/projects.py 89 96 92.71%
src/pylorax/api/v0.py 86 93 92.47%
src/composer/cli/sources.py 11 49 22.45%
Files with Coverage Reduction New Missed Lines %
src/pylorax/api/yumbase.py 1 96.1%
Totals Coverage Status
Change from base Build 230: 2.08%
Covered Lines: 2032
Relevant Lines: 4521

💛 - Coveralls

@bcl
Copy link
Contributor Author

bcl commented Jun 22, 2018

Merged

@bcl bcl closed this Jun 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants