Allow ansible-galaxy to install roles from URLs #8600

Closed
wants to merge 11 commits into
from

Conversation

Projects
None yet
4 participants
@willthames
Contributor

willthames commented Aug 13, 2014

ansible-galaxy can now refer to SCM URLs (git and hg at this point)
for role names
Dependencies need to use the full SCM URLs too.
Otherwise all seems to work well

Test rolesfile

http://bitbucket.org/willthames/git-ansible-galaxy,v1.1
https://bitbucket.org/willthames/hg-ansible-galaxy

(works with ssh too)

@mpdehaan

This comment has been minimized.

Show comment
Hide comment
@mpdehaan

mpdehaan Aug 13, 2014

Contributor

As posted to the ML, how does the code know a http:// and https:// URL is git or hg or svn?

I'm wondering if maybe we need to do git+https:// or something.

Contributor

mpdehaan commented Aug 13, 2014

As posted to the ML, how does the code know a http:// and https:// URL is git or hg or svn?

I'm wondering if maybe we need to do git+https:// or something.

@willthames

This comment has been minimized.

Show comment
Hide comment
@willthames

willthames Aug 13, 2014

Contributor

@mpdehaan see lines 334 onwards. It uses git ls-remote and hg identity - whichever has an exit status of 0 is the right SCM to use.

No real need for the git+https although might want to extend it to svn, bzr etc.

Contributor

willthames commented Aug 13, 2014

@mpdehaan see lines 334 onwards. It uses git ls-remote and hg identity - whichever has an exit status of 0 is the right SCM to use.

No real need for the git+https although might want to extend it to svn, bzr etc.

@willthames

This comment has been minimized.

Show comment
Hide comment
@willthames

willthames Aug 14, 2014

Contributor

Changed to use git+http as it does make determining SCM simpler and more explicit.

I can squash commits before this is accepted but wanted to preserve history for now in case this needs further revision.

Contributor

willthames commented Aug 14, 2014

Changed to use git+http as it does make determining SCM simpler and more explicit.

I can squash commits before this is accepted but wanted to preserve history for now in case this needs further revision.

@mpdehaan mpdehaan added P2 labels Aug 14, 2014

willthames added some commits Aug 13, 2014

Allow ansible-galaxy to install roles from URLs
ansible-galaxy can now refer to SCM URLs (git and hg at this point)
for role names
Dependencies need to use the full SCM URLs too.
Otherwise all seems to work well

Test rolesfile
```
http://bitbucket.org/willthames/git-ansible-galaxy,v1.1
https://bitbucket.org/willthames/hg-ansible-galaxy
```
(works with ssh too)
Change to how SCM is determined
Change SCM determination from executing git/hg commands to explicit
in URL.

Fix check for already installed dependencies
Need to take URLs into account when running playbooks too
Ensure that if meta/main.yml has dependencies containing URLs
these are correctly resolved at playbook runtime.

Update role_name during run time so that output of e.g. task names
look sensible
Tidied up a little, added tests
Moved repo_url_to_role_name to common method in ansible.utils
Added unit test for repo_url_to_role_name
Added integration tests for galaxy
Friendly Role Names and roles from URLs
* Roles can now be given a friendly name as third field in role spec csv
* Roles can be installed from URL (not just from archived SCMs)
* Integration tests to demonstrate this
* Unit tests to ensure that role spec parsing works as expected
Allow installation of roles from yaml roles file
Added docs
Added more tests
Improved how roles are returned from the parsers
Handle older git archive commands
Older git archive commands create tar archives even with a tar.gz
extension. So change it to always create tar archives and have
the install_role method cope.

Removed ssh roles from the test case as they don't work unless
you can connect to bitbucket via ssh and have your key there.

Corrected a minor typo in error messages
Make galaxy work when API server not available
`ansible-galaxy init --offline ...` can create a role without
talking to the galaxy api server
`ansible-galaxy install ...` only needs to talk to the galaxy api
server for galaxy roles, not tar files or scm archives
Fixed a bug in command line role installation
Corrected bug where role_path was wrong for roles in subdirectories
Fixed role name for
    - { role: 'lives/in/a/subdirectory' }
Should be 'lives/in/a/subdirectory', not just 'subdirectory'
@mpdehaan

This comment has been minimized.

Show comment
Hide comment
@mpdehaan

mpdehaan Aug 21, 2014

Contributor

Checking this out now...

Contributor

mpdehaan commented Aug 21, 2014

Checking this out now...

@mpdehaan

This comment has been minimized.

Show comment
Hide comment
@mpdehaan

mpdehaan Aug 21, 2014

Contributor

Awesome, I've merged this, also adding a docs section to the ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the system understand that github.com URLs are git+ protocol in a few cases, as I originally found the "non-tarball" message a little confusing and wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference docs about these options.

Thank you very much this will be a very very popular feature!

Contributor

mpdehaan commented Aug 21, 2014

Awesome, I've merged this, also adding a docs section to the ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the system understand that github.com URLs are git+ protocol in a few cases, as I originally found the "non-tarball" message a little confusing and wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference docs about these options.

Thank you very much this will be a very very popular feature!

@mpdehaan

This comment has been minimized.

Show comment
Hide comment
@mpdehaan

mpdehaan Aug 21, 2014

Contributor

Closing this, I'll reply to the list discussion in a bit.

Thanks!

Contributor

mpdehaan commented Aug 21, 2014

Closing this, I'll reply to the list discussion in a bit.

Thanks!

@mpdehaan mpdehaan closed this Aug 21, 2014

@ibratoev

This comment has been minimized.

Show comment
Hide comment
@ibratoev

ibratoev Aug 21, 2014

it already is ;)

On Fri, Aug 22, 2014 at 12:14 AM, Michael DeHaan notifications@github.com
wrote:

Awesome, I've merged this, also adding a docs section to the
ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the
system understand that github.com URLs are git+ protocol in a few cases,
as I originally found the "non-tarball" message a little confusing and
wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference
docs about these options.

Thank you very much this will be a very very popular feature!


Reply to this email directly or view it on GitHub
#8600 (comment).

it already is ;)

On Fri, Aug 22, 2014 at 12:14 AM, Michael DeHaan notifications@github.com
wrote:

Awesome, I've merged this, also adding a docs section to the
ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the
system understand that github.com URLs are git+ protocol in a few cases,
as I originally found the "non-tarball" message a little confusing and
wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference
docs about these options.

Thank you very much this will be a very very popular feature!


Reply to this email directly or view it on GitHub
#8600 (comment).

@willthames

This comment has been minimized.

Show comment
Hide comment
@willthames

willthames Aug 21, 2014

Contributor

Awesome, that's great news.

I haven't had a look yet but on your note regarding github, it should be
able to handle downloading from a github URL pointing to an archive rather
than a repo. I think there's an integration test for that.

Will
On 22/08/2014 7:14 am, "Michael DeHaan" notifications@github.com wrote:

Awesome, I've merged this, also adding a docs section to the
ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the
system understand that github.com URLs are git+ protocol in a few cases,
as I originally found the "non-tarball" message a little confusing and
wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference
docs about these options.

Thank you very much this will be a very very popular feature!


Reply to this email directly or view it on GitHub
#8600 (comment).

Contributor

willthames commented Aug 21, 2014

Awesome, that's great news.

I haven't had a look yet but on your note regarding github, it should be
able to handle downloading from a github URL pointing to an archive rather
than a repo. I think there's an integration test for that.

Will
On 22/08/2014 7:14 am, "Michael DeHaan" notifications@github.com wrote:

Awesome, I've merged this, also adding a docs section to the
ansible-galaxy docs page.

I've also upgraded error handling in a few minor places, and made the
system understand that github.com URLs are git+ protocol in a few cases,
as I originally found the "non-tarball" message a little confusing and
wanted to fix that up.

This works solidly and I greatly appreciate the tests.

Docs should appear soon and I'll also write the list about this.

We also need to update the galaxy website eventually to cross-reference
docs about these options.

Thank you very much this will be a very very popular feature!


Reply to this email directly or view it on GitHub
#8600 (comment).

@willthames willthames deleted the willthames:galaxy_scm_roles branch Aug 22, 2014

@willthames

This comment has been minimized.

Show comment
Hide comment
@willthames

willthames Aug 22, 2014

Contributor

Created #8714 for the github archive problem.

Contributor

willthames commented Aug 22, 2014

Created #8714 for the github archive problem.

@ansibot ansibot added feature and removed feature_pull_request labels Mar 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment