feat(pipelines): support workload deployment orders in pipelines

[efekarakus]

Resolves #3402

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

[iamhopaul123]

I might be stupid but I am a little confused here 🥲 Only when bfs.marked[neighbor] is false can the program proceed here, but then does that mean bfs.ranks[neighbor] will be empty too?

I feel like if a vertex is visited, we shouldn't skip too quickly but possibly think about if we should update the rank and update with the lowest rank rather than the highest. For example:

a -> b -> c
d -> c -> f

where both a and d has 0 rank. If we give c rank 2 rather than 1 then f will share the same rank as c.

[efekarakus]

lol it's not stupid -

but then does that mean bfs.ranks[neighbor] will be empty too?

I think your example shows how it's possible that ranks is not empty. In your example, after a -> b -> c we will have:

map[string]int {
    "a": 0,
    "b": 1,
    "c": 2
}

Then when we start from the second root, d -> c -> f we do not want the rank of c to become 1. It has a higher rank from a previous deployment, so we want to keep it at 2. The final ranks from the algorithm will be:

map[string]int {
   "a": 0,
   "d": 0,
   "b": 1,
   "c": 2,
   "f": 3,
}

We can never set c to have rank 1 otherwise it will run in parallel with b which should happen prior to c. Hope this makes sense!

[iamhopaul123]

Yeah now I agree we should set c to 2. However, if we start from d then since c is visited, it won't get updated to a higher rank 2

[efekarakus]

it will boss because of this if statement.
Since each time traverse is called marked is reset. After starting from d we will have:

ranks: {"d": 0, "c": 1, f: "2"}

Then we reset all marks so c is no longer marked.
When we start from a, in this if-statement we get:

if rank /* 2 */ := bfs.ranks[vtx] /* rank of "b" = 1 */ + 1; rank > bfs.ranks[neighbor ] /*  2 > 1 so update ranks["c"] */

[dannyrandall]

😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋😋

[efekarakus]

boss has a good point where the wrong rank will be calculated for the following scenario:

a -> f  -> d
a -> b -> c -> d

in this scneario, d will have rank 2 instead of 3. So added the label until this test case and a modification is submitted

[iamhopaul123]

LGTM just a nit 🚢

[iamhopaul123]

Suggested change

	indegrees map[V]int // Number of incoming edges for each vertex.
	inDegrees map[V]int // Number of incoming edges for each vertex.

[dannyrandall]

lgtm!

[dannyrandall]

nit

Suggested change

	arr := make([]V, len(neighbors))
	i := 0
	for neighbor := range neighbors {
	arr[i] = neighbor
	i += 1
	}
	var arr []V
	for neighbor := range neighbors {
	arr = append(arr, neighbor)
	}