fix(bake): Lookup artifact details from all upstream stages

[marchello2000]

Imagine following setup:

Pipeline A
 -> Jenkins job (produces foo.*.deb)
 -> Run Pipeline B
    -> Jenkins job (produces bar.*.deb)
    -> Bake package foo

In this case, the package lookup for bake stage will fail to lookup artifact
details for foo.deb to pass to the bakery. Thus the bakery is free to pick the
latest artifact matching the name which can be the wrong artifact.

This change allows package look up to traverse up parent pipeline stages, similar
to FindImage tasks

[marchello2000]

i tried doing Comparator.comparingNullLast(Comparator.comparing(stage::getEndTime)) but it was throwing NPE, if you know what I was doing wrong, would love to simplify this to use an off the shelf comparator

[aravindmd]

do we have any context on why this time comparison is required ?

[marchello2000]

this is a great mystery... hence my comment about this being a "port to java". I wonder if it's there for unit tests?
Or, more likely, this is a relic of the past that could be removed? @ajordens / @duftler do either of you happen to know the history here? (#874)

[spinnakerbot]

Please delete the pull request instructions from the body of your pull request message.

The instructions start with the line:

We prefer small, well tested pull requests.

You can reopen your pull request when this has been addressed.

[emjburns]

This goes only one level back, correct? So your package must be created either in the existing pipeline or its parent, but not its grandparent.

[aravindmd]

can we add a test case for this case if not present already : parent -> child -> grandchild

[asher]

I think this will go back multiple levels, getAncestors() recurses:

      if (parentPipelineStage.isPresent()) {
        allPriorStages.addAll(getAncestors(parentPipelineStage.get(), parentPipelineExecution));
      } else {

This could be problematic from a resource consumption standpoint, especially in cases I've seen within the community where users have massive chains of dependent pipelines.

I'd like to see parent pipeline traversal limited to fulfilling a search condition. So instead of "get all stages across all parents, then filter for $condition", "traverse ancestors until $condition".

[marchello2000]

Yes, @asher is correct - this should recurse until root execution is found.

@asher : can you elaborate on this comment (we can chat in person as well - no rush). I agree it would be more efficient to only recurse until the condition is satisfied. But other than the efficiency are there any other reasons for it that I am not understanding?

Because the stages are returned in order (e.g. closest ancestor first) so the search downstream will find the closest match (assuming they traverse in order which currently all callers do).

To be clear: I like your suggestion and I think it will actually simplify some findImage code - just want to make sure I understand it completely since I am not super familiar with various pipeline designs you've seen from the community

[asher]

I think this will go back multiple levels, getAncestors() recurses:

      if (parentPipelineStage.isPresent()) {
        allPriorStages.addAll(getAncestors(parentPipelineStage.get(), parentPipelineExecution));
      } else {

This could be problematic from a resource consumption standpoint, especially in cases I've seen within the community where users have massive chains of dependent pipelines.

I'd like to see parent pipeline traversal limited to fulfilling a search condition. So instead of "get all stages across all parents, then filter for $condition", "traverse ancestors until $condition".

[marchello2000]

@asher : made the function take a predicate and terminate as soon as match is found - a little more code but the consumption side nicer (not to mention it's more efficient, thanks!)
Also added some "grandparent" tests

[asher]

Looks good, thanks for implementing the predicate based approach!