Dependent builds #249

Closed
dkubb opened this Issue Sep 7, 2011 · 23 comments

Comments

Projects
None yet
@dkubb

dkubb commented Sep 7, 2011

An awesome feature to have would be a way to specify project dependencies (or better yet infer them from the Gemfile/gemspec), so that when a project builds successfully, all it's dependents' (that travis knows about) are built too.

This is a feature we use in the DataMapper CI, where anytime dm-core passes, it executes a build for every other dependent gem.

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Sep 7, 2011

Contributor

Sounds like a cool idea to ensure continual integration ;-)

How would we update those dependencies to reference the one we just built?

Contributor

parndt commented Sep 7, 2011

Sounds like a cool idea to ensure continual integration ;-)

How would we update those dependencies to reference the one we just built?

@dkubb

This comment has been minimized.

Show comment
Hide comment
@dkubb

dkubb Sep 7, 2011

I don't know if travis aims to be general purpose, or if it's going to be only ruby specific, so this approach may not apply. I'm also not sure if travis is coupled to bundler either, but if it is I think this might work.

I was thinking it might be possible to track the dependencies in each Gemfile.lock. When one of the dependencies' builds are successful, we go to each dependent, run the "bundle update" and if the Gemfile.lock changes we rebuild the gem.

In other words we leverage bundler to tell us when a gem's dependencies have changed rather than trying to do too much inside travis.

dkubb commented Sep 7, 2011

I don't know if travis aims to be general purpose, or if it's going to be only ruby specific, so this approach may not apply. I'm also not sure if travis is coupled to bundler either, but if it is I think this might work.

I was thinking it might be possible to track the dependencies in each Gemfile.lock. When one of the dependencies' builds are successful, we go to each dependent, run the "bundle update" and if the Gemfile.lock changes we rebuild the gem.

In other words we leverage bundler to tell us when a gem's dependencies have changed rather than trying to do too much inside travis.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Dec 19, 2011

Member

This is a cool feature which we nick named 'dependent builds'. The easiest and most reliable way to do this is to specify repos to test in the .travis.yml, but those repos would need to use the master version of the changed repo, if you know what I mean :)

We would love someone to help with this feature :)

Member

joshk commented Dec 19, 2011

This is a cool feature which we nick named 'dependent builds'. The easiest and most reliable way to do this is to specify repos to test in the .travis.yml, but those repos would need to use the master version of the changed repo, if you know what I mean :)

We would love someone to help with this feature :)

@svenfuchs

This comment has been minimized.

Show comment
Hide comment
@svenfuchs

svenfuchs Feb 13, 2012

Member

Changing the title to "Dependent builds"

Member

svenfuchs commented Feb 13, 2012

Changing the title to "Dependent builds"

@fd

This comment has been minimized.

Show comment
Hide comment
@fd

fd May 31, 2012

Any news on this?

fd commented May 31, 2012

Any news on this?

@letmaik

This comment has been minimized.

Show comment
Hide comment
@letmaik

letmaik Nov 12, 2012

This is important for Maven projects in Java, too. But it applies only when a project depends on SNAPSHOT versions. For new release versions, a change in the pom must be done and committed which then triggers a build anyway.

letmaik commented Nov 12, 2012

This is important for Maven projects in Java, too. But it applies only when a project depends on SNAPSHOT versions. For new release versions, a change in the pom must be done and committed which then triggers a build anyway.

@sarahhodne

This comment has been minimized.

Show comment
Hide comment
@sarahhodne

sarahhodne Dec 14, 2012

Contributor

Which way should the specification go? "If this repository passes, build these other repos" or vice versa? The latter sounds more sensible to me.

However, if we go with that we need to store this information somehow, as we can't realistically go through every GitHub repo with a .travis.yml file checking if they depend on this build.

We could add a Dependency model that hooks up two repositories, but then we need to make sure that this stays in sync every time the configuration updates.

As @joshk said, we also need to figure out what version of the dependency is the "master". This could be as easy as specifying a branch together with the name of the repository when specifying dependencies, though.

If anyone else has any ideas on this, please do tell.

Contributor

sarahhodne commented Dec 14, 2012

Which way should the specification go? "If this repository passes, build these other repos" or vice versa? The latter sounds more sensible to me.

However, if we go with that we need to store this information somehow, as we can't realistically go through every GitHub repo with a .travis.yml file checking if they depend on this build.

We could add a Dependency model that hooks up two repositories, but then we need to make sure that this stays in sync every time the configuration updates.

As @joshk said, we also need to figure out what version of the dependency is the "master". This could be as easy as specifying a branch together with the name of the repository when specifying dependencies, though.

If anyone else has any ideas on this, please do tell.

@svenfuchs

This comment has been minimized.

Show comment
Hide comment
@svenfuchs

svenfuchs Dec 14, 2012

Member

I think the generalized idea here is some kind of "subscription" api. I.e. "one" (an app) can subscribe to events on builds. Now this app can hold the information that

  • travis-core wants to trigger travis-api (case 1, here one's in control of dependencies)
  • travis-api wants to be triggered when sinatra has built (case 2, not in control of sinatra)

... and in turn hit the travis-api to run a build.

We'd also want to consider how we allow to filter things and how we allow to parametrize the created build. E.g. it would be handy to be able to run a feature branch on travis-api when a feature branch on travis-core has passed.

my2c :)

Member

svenfuchs commented Dec 14, 2012

I think the generalized idea here is some kind of "subscription" api. I.e. "one" (an app) can subscribe to events on builds. Now this app can hold the information that

  • travis-core wants to trigger travis-api (case 1, here one's in control of dependencies)
  • travis-api wants to be triggered when sinatra has built (case 2, not in control of sinatra)

... and in turn hit the travis-api to run a build.

We'd also want to consider how we allow to filter things and how we allow to parametrize the created build. E.g. it would be handy to be able to run a feature branch on travis-api when a feature branch on travis-core has passed.

my2c :)

@themihai

This comment has been minimized.

Show comment
Hide comment

+1

@themihai

This comment has been minimized.

Show comment
Hide comment
@themihai

themihai Dec 26, 2013

I would propose to add a git submodule detection so when a submodule is detected the build plan should start watching these repositories from submodules as well.
Simply said when any of the repositories loaded as submodules are updated the build plan should get triggered as well.
Another option would be to simply define the repositories you want to watch as dependencies on .travis.yml .

I would propose to add a git submodule detection so when a submodule is detected the build plan should start watching these repositories from submodules as well.
Simply said when any of the repositories loaded as submodules are updated the build plan should get triggered as well.
Another option would be to simply define the repositories you want to watch as dependencies on .travis.yml .

@jbohren

This comment has been minimized.

Show comment
Hide comment
@jbohren

jbohren Dec 30, 2013

Another option is to just make this part of the GitHub integration. Right now the service hooks let you specify a travis user/token/domain, but in addition there could be a "dependencies" list where a new build for that repo gets triggered if a specified branch of another specified repo builds and passes.

jbohren commented Dec 30, 2013

Another option is to just make this part of the GitHub integration. Right now the service hooks let you specify a travis user/token/domain, but in addition there could be a "dependencies" list where a new build for that repo gets triggered if a specified branch of another specified repo builds and passes.

@analytically

This comment has been minimized.

Show comment
Hide comment

+1

@lkrnac

This comment has been minimized.

Show comment
Hide comment

lkrnac commented Apr 14, 2014

+1

@kyab

This comment has been minimized.

Show comment
Hide comment

kyab commented May 20, 2014

+1

@ioquatix

This comment has been minimized.

Show comment
Hide comment

👍

@murrekatt

This comment has been minimized.

Show comment
Hide comment
@murrekatt

murrekatt Jul 4, 2014

@themihai, submodules work differently. using a submodule you have already pinned the repo with that submodule to use a specific commit of the submodule. if you want to change this you would manually adjust that with e.g. git pull and then commit that. this will trigger a rebuild of the repo. hence, all good.

the "problem" comes when you don't control the version explicitly inside the repo, but rather rely on a mechanism outside like follow HEAD of another repo or e.g. a tag that is moved somewhere. in both these cases, the repo you want to rebuild doesn't know that a dependency changed unless it is told so.

one way to provide a generic solution to this would be to add an explicit list of github repos in the .travis.yml file. this could allow for e.g. automatic setup of webhooks or other triggers from those repos or using a subscription api to changes in those repos.

in any case, this is an important ci feature and needs to be repo content agnostic. it's a trigger from repo/branch to repo/branch. i.e. if .travis.yml would say it depends on a repo r and only branch b, changes to only that combination will trigger.

what would prevent this kind of setup for travis?

@themihai, submodules work differently. using a submodule you have already pinned the repo with that submodule to use a specific commit of the submodule. if you want to change this you would manually adjust that with e.g. git pull and then commit that. this will trigger a rebuild of the repo. hence, all good.

the "problem" comes when you don't control the version explicitly inside the repo, but rather rely on a mechanism outside like follow HEAD of another repo or e.g. a tag that is moved somewhere. in both these cases, the repo you want to rebuild doesn't know that a dependency changed unless it is told so.

one way to provide a generic solution to this would be to add an explicit list of github repos in the .travis.yml file. this could allow for e.g. automatic setup of webhooks or other triggers from those repos or using a subscription api to changes in those repos.

in any case, this is an important ci feature and needs to be repo content agnostic. it's a trigger from repo/branch to repo/branch. i.e. if .travis.yml would say it depends on a repo r and only branch b, changes to only that combination will trigger.

what would prevent this kind of setup for travis?

@ajdm

This comment has been minimized.

Show comment
Hide comment
@ajdm

ajdm Aug 4, 2014

It doesn't solve this scenario, but this script solves the inverse problem, namely rebuilding projects dependent on a parent project that you own.

ajdm commented Aug 4, 2014

It doesn't solve this scenario, but this script solves the inverse problem, namely rebuilding projects dependent on a parent project that you own.

@adrianbk

This comment has been minimized.

Show comment
Hide comment
@adrianbk

adrianbk Dec 21, 2014

The following gradle plugin uses the travisci api to build dependant/child projects https://github.com/adrianbk/gradle-travisci-trigger-plugin

The following gradle plugin uses the travisci api to build dependant/child projects https://github.com/adrianbk/gradle-travisci-trigger-plugin

@hugovk hugovk referenced this issue in python-pillow/Pillow Jan 1, 2015

Closed

Run OS X tests more regularly #1065

3 of 3 tasks complete
@kbrock

This comment has been minimized.

Show comment
Hide comment
@kbrock

kbrock Mar 3, 2015

This reminds me a little of automatic publishing github pages - the difference being that it would just trigger a build in another repo.

Would it be possible to leverage a technique like this?

kbrock commented Mar 3, 2015

This reminds me a little of automatic publishing github pages - the difference being that it would just trigger a build in another repo.

Would it be possible to leverage a technique like this?

@nickmccurdy

This comment has been minimized.

Show comment
Hide comment
@nickmccurdy

nickmccurdy Mar 7, 2015

👍 for dependent builds.

👍 for dependent builds.

@doctorrustynelson

This comment has been minimized.

Show comment
Hide comment

@travis-ci travis-ci locked and limited conversation to collaborators Mar 20, 2015

@BanzaiMan

This comment has been minimized.

Show comment
Hide comment
@BanzaiMan

BanzaiMan Mar 20, 2015

Member

This is on our road map. If you want to be notified of progress when it happens, please subscribe, not 👍.

Member

BanzaiMan commented Mar 20, 2015

This is on our road map. If you want to be notified of progress when it happens, please subscribe, not 👍.

@roidrage

This comment has been minimized.

Show comment
Hide comment
@roidrage

roidrage Jul 23, 2015

Member

Closing this issue for now, as we have no immediate plans to add this feature.

Should we add it to the roadmap eventually, we'll make sure to update this ticket.

Member

roidrage commented Jul 23, 2015

Closing this issue for now, as we have no immediate plans to add this feature.

Should we add it to the roadmap eventually, we'll make sure to update this ticket.

@roidrage roidrage closed this Jul 23, 2015

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.