Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't re-run all Tasks when re-running a Job #157

Closed
kavinderd opened this issue Oct 2, 2015 · 4 comments
Closed

Don't re-run all Tasks when re-running a Job #157

kavinderd opened this issue Oct 2, 2015 · 4 comments
Labels
efficiency

Comments

@kavinderd
Copy link

As discussed on the slack channel last week. A performance improvement that would be really nice is if tasks within a job could be cached. So if I make a change to something affecting Task 3 in Job A, Task 1 & 2 don't need to be re-run. A lot of our jobs involve making a docker image, then testing some property of it and then pushing it somewhere, and we're usually only iterating on the second half of our tasks.
@concourse-bot
Copy link
Collaborator

concourse-bot commented Oct 2, 2015 •
edited
Loading

Hi there!

We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.

The current status is as follows:

  • #140235667 Don't re-run all Tasks when re-running a Job
  • #104768192 I can re-run a specific step in a build

This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.

@vito vito mentioned this issue Jan 13, 2016
Closed
@jvshahid
Copy link
Contributor

jvshahid commented Apr 8, 2016

I really like this idea. I think this should be an optional config option per task. This will avoid having concourse understand how the task script run, for example if it depends on current time or external services.

@vito
Copy link
Member

vito commented Jul 27, 2016

This is much more doable/useful with the new container retention semantics, wherein all containers for a failed build are kept as long as it's the current build.

cdutra pushed a commit that referenced this issue Aug 4, 2016
bump atc fly testflight ginkgo gomega
ebee7fc 
[finishes #127219723]

Submodule src/github.com/concourse/atc 1ee1c48..a3023dd:
  > ginkgo blur reoreded everything
  > return all public pipelines on GetAllPipelines endpoint
  > only check for basic auth on get token endpoint
  > do not default team name to main if not provided
  > do not default team name to main in api
Submodule src/github.com/concourse/fly cb98e4c..3269297:
  > make --team-name required
  > retrieves token if there is no auth method set
Submodule src/github.com/concourse/testflight fdd0c3f..abd306e:
  > do not default team name to 'main'
Submodule src/github.com/onsi/ginkgo e43390e..74c678d:
  > Make JUnit reporter include failure location in message. (#262)
  > remove 1.4 from travis.yml
  > Add gcflags option (#276)
  > Revert "Use the go1.5 build tag to handle vendor exceptions" (#274)
  > Merge pull request #272 from fsouza/fix-vendor
  > Add flaky test mitigation (#261)
  > Allow units and precision in benchmark (#266)
  > Add Solaris support (#264)
  > Merge pull request #259 from kwadrat/master
  > Merge branch 'apvail-spell-fix'
  > Fix go16 vendor
  > Merge pull request #250 from james-lawrence/master
  > Merge pull request #228 from jayunit100/RegexFileNameFiltering
  > Fix test flakiness
  > Merge pull request #235 from mboersma/fix-travis
  > fix compilation on older versions of go
  > fix issue where packages that reference vendored dependencies weren't compiling
  > Merge pull request #216 from sha1sum/master
  > Merge pull request #209 from luxas/build_on_arm64
  > Merge pull request #212 from cfmobile/master
  > Merge pull request #210 from cfmobile/master
Submodule src/github.com/onsi/gomega 2152b45..9ed8da1:
  > Merge pull request #166 from trayo/patch-2
  > Merge pull request #164 from wendorf/assert_typo
  > Merge remote-tracking branch 'origin/pr/163'
  > Merge pull request #160 from tinygrasshopper/fix_failing_close_ghttp
  > Merge pull request #150 from tinygrasshopper/build-fix
  > Merge pull request #159 from WesleyJeanette/patch-1
  > Merge pull request #157 from kwadrat/master
  > Merge pull request #141 from mariantalla/gomega-yaml-matcher
  > Reset tmpDir in gexec.CleanupBuildArtifacts
  > Update test description for match json tests.
  > Make the error message for expected JSON values having the wrong type accurate
  > Merge pull request #133 from tjarratt/be-identical-to-matcher
  > Merge pull request #132 from tjarratt/improve-match-json-error-message
  > Merge pull request #128 from tinygrasshopper/have-cap
  > drop 1.4 from travis
  > ghttp tests should now pass in 1.6
  > CloseClientConnections test uses http.Post instead of http.Get to avoid retries
  > add tip to .travis.yml
  > Merge pull request #125 from cfmobile/master
  > Merge pull request #122 from cfmobile/master
  > Merge pull request #119 from jim-slattery-rs/gitignore_idea
  > Merge pull request #118 from jim-slattery-rs/fix_up_succeed

Signed-off-by: Yucheng Tu <ytu@pivotal.io>
clarafu added a commit that referenced this issue Mar 28, 2017
@clarafu @joshzarrabi
bump atc
33eaac7 
Submodule src/github.com/concourse/atc d30b216..8618312:
  > Merge pull request #154 from jfredrickson5/master
  > Merge pull request #157 from jmcarp/patch-1

Signed-off-by: Josh Zarrabi <jzarrabi@pivotal.io>
@vito vito removed the enhancement label Nov 28, 2017
@vito
Copy link
Member

vito commented Nov 28, 2017

I don't see us targeting this feature in the forseeable future, but if someone wants to chime in with more details in a new proposal, feel free. This sounds like a "nice to have" but the devil will be in the details.

@vito vito closed this as completed Nov 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
efficiency
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@vito @jvshahid @kavinderd @chendrix @concourse-bot