Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix upstream merge conflicts [DO NOT MERGE, just review] #62

Conversation

npazosmendez
Copy link

Original PR: #60

This PR has the same changes but with a different workflow to make review easier. This is what I did:

  1. Created the branch npazosmendez/sync-upstream2 and merged the upstream changes there. Commited everything blindly, with conflicts included.
  2. From npazosmendez/sync-upstream2 I branched to npazosmendez/sync-upstream-fix-conflicts, where I compressed all the fixes to the conflicts in a single commit.
  3. Created this PR that merges npazosmendez/sync-upstream2 <-- npazosmendez/sync-upstream-fix-conflicts . This makes the diff of this PR just have the fixes to the conflicts and nothing else

Once this PR is approved we can then merge npazosmendez/sync-upstream2 into main.

Note that I'm still missing some fixes. Those will come in separate commits, but the diff should still be clean.

@ywwg
Copy link

ywwg commented Aug 9, 2022

as said in slack:

The problem is this will either:

  1. squash down so our changes and the upstream merge will be in one commit, which will make future merges from HEAD hard (because the history will not be shared)
  2. if unsquashed, there will be completely broken commits with the conflict markers

I think we need to merge in from HEAD unsquashed I think, and do the fixes in the single merge commit

@grafanabot

This comment has been minimized.

@jesusvazquez
Copy link
Member

jesusvazquez commented Aug 9, 2022

Maybe it helps to know how we keep https://github.com/grafana/mimir-prometheus/ up to date with github.com/prometheus/prometheus/

When we want to update mimir-prometheus with upstream changes, a separate PR is open, for example grafana/mimir-prometheus#287 This PR is always merged with a Merge commit not a squash.

Then any changes performed on top of the mimir-prometheus fork are merged from separate prs and they can either be squashed commits or merged commits but usually they prefer squashed commits to keep 1 commit 1 pr parity.

The benefits of this workflow is that you can always compare the fork to upstream and cherry pick any of the changes done in the fork in case you want to donate them back to upstream or just eventually sync up with upstream.

@npazosmendez
Copy link
Author

npazosmendez commented Aug 9, 2022

@jesusvazquez are always all the merge conflicts fixes contained in the Merge commit? Or can those conflicts be resolved incrementally?

@npazosmendez
Copy link
Author

Ok this is what I'm thinking:

  1. We use this PR for reviewing
  2. Once approved, I use the diff npazosmendez/sync-upstream2 vs npazosmendez/sync-upstream-fix-conflicts to create a patch that solves all the conflicts.
  3. In a new branch X, I do the merge from scratch and use the patch to solve the conflicts. Then commit the Merge.
  4. We open a new PR main <-- X and merge it without squash to keep all the commits

@jesusvazquez
Copy link
Member

@jesusvazquez are always all the merge conflicts fixes contained in the Merge commit? Or can those conflicts be resolved incrementally?

Any conflicts that may surface are addressed in the same PR as separate commits and then the entire PR is merged not squashed.

@npazosmendez npazosmendez marked this pull request as ready for review August 10, 2022 14:51
Copy link

@ywwg ywwg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ran out of steam but here are my comments so far

ret.Tags["name"] = metric
ret.PathExpression = ret.Name
ret.Values = a.Values
=======
name := part[len(part)-1]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we not copying the tags or values?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those deleted lines were in our HEAD. Upstream deleted those and added the ret := a.CopyName(name) a few lines below: msaf1980@cf1f002

(this is the problem with this reviewing approach, the upstream changes are a little hidden. But I think overall it's better than a ~300 files PR)

Want: []*types.MetricData{types.MakeMetricData("foo==.baz",
[]float64{1, 2, 3, 4, 5}, 1, now32)},
},
// FIXME: I don't think `metric1.foo==.bar.baz` is a valid metric name, that is making this test fail
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please attach an Issue number to this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

},
},
// FIXME: I don't think `..foo==.bar` is a valid metric name, that is making these testsapi/cmd/carbonapi/ht fail
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ibid, same issue as above

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

r := a.CopyLink()
r.Name = tempName
r.Tags["name"] = r.Name
=======
r := a.CopyName(tempName)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does CopyName do the same changes that CopyLink did?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r := *a
r.Name = "nPercentile(" + a.Name + "," + percentStr + ")"
>>>>>>> upstream/main
r := a.CopyLink()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some places, CopyName is used, and in some, CopyLink is used. What is the difference?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CopyName is a new helper function the upstream introduced, check the behavior here: https://github.com/grafana/carbonapi/blob/npazosmendez/sync-upstream-fix-conflicts/expr/types/types.go#L452

I think it could be avoided. (there's lots of helpers that do almost the same thing (and sometimes the same thing): Copy, CopyLink, CopyName, CopyTag, CopyLinkTags, etc etc etc).

But I think changing that is out of the scope of this PR, wanted to leave the upstream changes as clean as possible.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to having CopyName be available! There's a lot of places where I've seen something like this happening:

md := origMd.CopyLink()
md.Name = "new name"
...

The problem with that is that the "name" tag then doesn't get updated correctly, but it looks like CopyName covers that.

@@ -2,6 +2,7 @@ package removeBelowSeries

import (
"context"
"fmt"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this import being used?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The merge deleted it and I had to add it again to fix the build. The build would fail if it wasn't being used anyway.

@@ -2,6 +2,7 @@ package smartSummarize

import (
"context"
"fmt"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the import being used?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@grafanabot
Copy link

Go coverage report:

Click to expand.
File %
github.com/grafana/carbonapi/cmd/carbonapi/http/capability.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/enrichcontext.go 6.7%
github.com/grafana/carbonapi/cmd/carbonapi/http/find_handlers.go 33.5%
github.com/grafana/carbonapi/cmd/carbonapi/http/functions_handler.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/helper.go 27.3%
github.com/grafana/carbonapi/cmd/carbonapi/http/info_handlers.go 53.8%
github.com/grafana/carbonapi/cmd/carbonapi/http/init.go 84.6%
github.com/grafana/carbonapi/cmd/carbonapi/http/lbcheck_handler.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/metrics.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/render_handler.go 50.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/tags_handler.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/usage_handler.go 0.0%
github.com/grafana/carbonapi/cmd/carbonapi/http/version_handler.go 0.0%
github.com/grafana/carbonapi/date/date.go 78.9%
github.com/grafana/carbonapi/expr/consolidations/consolidations.go 44.5%
github.com/grafana/carbonapi/expr/expr.go 30.0%
github.com/grafana/carbonapi/expr/functions/absolute/function.go 80.0%
github.com/grafana/carbonapi/expr/functions/aggregate/function.go 82.2%
github.com/grafana/carbonapi/expr/functions/aggregateLine/function.go 81.0%
github.com/grafana/carbonapi/expr/functions/aggregateSeriesLists/function.go 73.5%
github.com/grafana/carbonapi/expr/functions/aggregateWithWildcards/function.go 86.7%
github.com/grafana/carbonapi/expr/functions/alias/function.go 83.3%
github.com/grafana/carbonapi/expr/functions/aliasByBase64/function.go 88.6%
github.com/grafana/carbonapi/expr/functions/aliasByMetric/function.go 92.9%
github.com/grafana/carbonapi/expr/functions/aliasByNode/function.go 84.2%
github.com/grafana/carbonapi/expr/functions/aliasByRedis/function.go 72.1%
github.com/grafana/carbonapi/expr/functions/aliasSub/function.go 80.0%
github.com/grafana/carbonapi/expr/functions/asPercent/function.go 77.6%
github.com/grafana/carbonapi/expr/functions/averageOutsidePercentile/function.go 89.3%
github.com/grafana/carbonapi/expr/functions/averageSeriesWithWildcards/function.go 89.1%
github.com/grafana/carbonapi/expr/functions/below/function.go 91.7%
github.com/grafana/carbonapi/expr/functions/cactiStyle/function.go 88.7%
github.com/grafana/carbonapi/expr/functions/changed/function.go 92.3%
github.com/grafana/carbonapi/expr/functions/constantLine/function.go 88.2%
github.com/grafana/carbonapi/expr/functions/delay/function.go 90.6%
github.com/grafana/carbonapi/expr/functions/derivative/function.go 93.8%
github.com/grafana/carbonapi/expr/functions/divideSeries/function.go 85.4%
github.com/grafana/carbonapi/expr/functions/ewma/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/exclude/function.go 81.8%
github.com/grafana/carbonapi/expr/functions/exp/function.go 90.9%
github.com/grafana/carbonapi/expr/functions/exponentialMovingAverage/function.go 72.2%
github.com/grafana/carbonapi/expr/functions/fallbackSeries/function.go 86.7%
github.com/grafana/carbonapi/expr/functions/filter/function.go 84.4%
github.com/grafana/carbonapi/expr/functions/grep/function.go 81.8%
github.com/grafana/carbonapi/expr/functions/groupByNode/function.go 86.5%
github.com/grafana/carbonapi/expr/functions/groupByTags/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/heatMap/function.go 86.4%
github.com/grafana/carbonapi/expr/functions/heatMap/helpers.go 84.6%
github.com/grafana/carbonapi/expr/functions/highestLowest/function.go 87.1%
github.com/grafana/carbonapi/expr/functions/hitcount/function.go 93.8%
github.com/grafana/carbonapi/expr/functions/identity/function.go 89.5%
github.com/grafana/carbonapi/expr/functions/integral/function.go 94.4%
github.com/grafana/carbonapi/expr/functions/integralByInterval/function.go 85.4%
github.com/grafana/carbonapi/expr/functions/integralWithReset/function.go 86.1%
github.com/grafana/carbonapi/expr/functions/interpolate/function.go 92.1%
github.com/grafana/carbonapi/expr/functions/invert/function.go 93.3%
github.com/grafana/carbonapi/expr/functions/isNotNull/function.go 93.8%
github.com/grafana/carbonapi/expr/functions/join/function.go 89.5%
github.com/grafana/carbonapi/expr/functions/keepLastValue/function.go 92.5%
github.com/grafana/carbonapi/expr/functions/legendValue/function.go 90.9%
github.com/grafana/carbonapi/expr/functions/limit/function.go 82.4%
github.com/grafana/carbonapi/expr/functions/linearRegression/function.go 80.0%
github.com/grafana/carbonapi/expr/functions/logarithm/function.go 90.9%
github.com/grafana/carbonapi/expr/functions/logit/function.go 91.3%
github.com/grafana/carbonapi/expr/functions/lowPass/function.go 89.3%
github.com/grafana/carbonapi/expr/functions/mapSeries/function.go 88.0%
github.com/grafana/carbonapi/expr/functions/minMax/function.go 86.7%
github.com/grafana/carbonapi/expr/functions/mostDeviant/function.go 88.6%
github.com/grafana/carbonapi/expr/functions/moving/function.go 82.3%
github.com/grafana/carbonapi/expr/functions/movingMedian/function.go 86.3%
github.com/grafana/carbonapi/expr/functions/multiplySeriesWithWildcards/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/nPercentile/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/nonNegativeDerivative/function.go 90.5%
github.com/grafana/carbonapi/expr/functions/offset/function.go 88.0%
github.com/grafana/carbonapi/expr/functions/offsetToZero/function.go 94.1%
github.com/grafana/carbonapi/expr/functions/pearson/function.go 86.1%
github.com/grafana/carbonapi/expr/functions/pearsonClosest/function.go 75.0%
github.com/grafana/carbonapi/expr/functions/perSecond/function.go 90.5%
github.com/grafana/carbonapi/expr/functions/percentileOfSeries/function.go 81.8%
github.com/grafana/carbonapi/expr/functions/polyfit/function.go 87.3%
github.com/grafana/carbonapi/expr/functions/pow/function.go 88.9%
github.com/grafana/carbonapi/expr/functions/powSeries/function.go 95.0%
github.com/grafana/carbonapi/expr/functions/rangeOfSeries/function.go 94.3%
github.com/grafana/carbonapi/expr/functions/removeBelowSeries/function.go 92.3%
github.com/grafana/carbonapi/expr/functions/removeBetweenPercentile/function.go 89.2%
github.com/grafana/carbonapi/expr/functions/removeEmptySeries/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/round/function.go 90.0%
github.com/grafana/carbonapi/expr/functions/scale/function.go 88.6%
github.com/grafana/carbonapi/expr/functions/scaleToSeconds/function.go 88.5%
github.com/grafana/carbonapi/expr/functions/seriesList/function.go 62.4%
github.com/grafana/carbonapi/expr/functions/sigmoid/function.go 91.3%
github.com/grafana/carbonapi/expr/functions/sinFunction/function.go 86.2%
github.com/grafana/carbonapi/expr/functions/slo/function.go 84.0%
github.com/grafana/carbonapi/expr/functions/slo/helpers.go 72.0%
github.com/grafana/carbonapi/expr/functions/smartSummarize/function.go 90.8%
github.com/grafana/carbonapi/expr/functions/sortBy/function.go 85.7%
github.com/grafana/carbonapi/expr/functions/sortByName/function.go 85.2%
github.com/grafana/carbonapi/expr/functions/squareRoot/function.go 90.5%
github.com/grafana/carbonapi/expr/functions/substr/function.go 85.4%
github.com/grafana/carbonapi/expr/functions/sumSeriesWithWildcards/function.go 88.9%
github.com/grafana/carbonapi/expr/functions/summarize/function.go 91.7%
github.com/grafana/carbonapi/expr/functions/timeShift/function.go 78.0%
github.com/grafana/carbonapi/expr/functions/timeShiftByMetric/function.go 89.7%
github.com/grafana/carbonapi/expr/functions/timeShiftByMetric/misc.go 93.8%
github.com/grafana/carbonapi/expr/functions/transformNull/function.go 88.7%
github.com/grafana/carbonapi/expr/functions/tukey/function.go 80.8%
github.com/grafana/carbonapi/expr/functions/unique/function.go 88.9%
github.com/grafana/carbonapi/expr/functions/weightedAverage/function.go 85.7%
github.com/grafana/carbonapi/expr/helper/align.go 60.7%
github.com/grafana/carbonapi/expr/helper/helper.go 0.0%
github.com/grafana/carbonapi/expr/helper/metric/extract.go 72.7%
github.com/grafana/carbonapi/expr/helper/sort.go 0.0%
github.com/grafana/carbonapi/expr/rewrite/aboveSeries/function.go 79.3%
github.com/grafana/carbonapi/expr/sort.go 96.1%
github.com/grafana/carbonapi/expr/tags/helper.go 100.0%
github.com/grafana/carbonapi/expr/types/extract.go 82.4%
github.com/grafana/carbonapi/expr/types/list.go 0.0%
github.com/grafana/carbonapi/expr/types/metricheap.go 0.0%
github.com/grafana/carbonapi/expr/types/types.go 33.9%
github.com/grafana/carbonapi/expr/types/windowed.go 0.0%
github.com/grafana/carbonapi/pkg/parser/define.go 76.9%
github.com/grafana/carbonapi/pkg/parser/interface.go 0.0%
github.com/grafana/carbonapi/pkg/parser/internal.go 9.4%
github.com/grafana/carbonapi/pkg/parser/interval.go 83.8%
github.com/grafana/carbonapi/pkg/parser/parser.go 44.3%
github.com/grafana/carbonapi/zipper/broadcast/broadcast_group.go 38.5%
github.com/grafana/carbonapi/zipper/helper/requests.go 13.3%
github.com/grafana/carbonapi/zipper/protocols/graphite/msgpack/type_gen.go 57.2%
github.com/grafana/carbonapi/zipper/protocols/irondb/irondb_group.go 2.5%
github.com/grafana/carbonapi/zipper/protocols/irondb/irondb_helpers.go 81.1%
github.com/grafana/carbonapi/zipper/protocols/prometheus/helpers/helpers.go 17.1%
github.com/grafana/carbonapi/zipper/protocols/victoriametrics/feature_set.go 19.7%
github.com/grafana/carbonapi/zipper/protocols/victoriametrics/fetch.go 0.0%
github.com/grafana/carbonapi/zipper/protocols/victoriametrics/find.go 0.0%
github.com/grafana/carbonapi/zipper/protocols/victoriametrics/tags.go 0.0%
github.com/grafana/carbonapi/zipper/protocols/victoriametrics/victoriametrics_group.go 7.8%
github.com/grafana/carbonapi/zipper/types/backend.go 0.0%
github.com/grafana/carbonapi/zipper/types/errors.go 0.0%
github.com/grafana/carbonapi/zipper/types/lbmethod.go 0.0%
github.com/grafana/carbonapi/zipper/types/requests.go 0.0%
github.com/grafana/carbonapi/zipper/types/response.go 6.4%
github.com/grafana/carbonapi/zipper/types/stats.go 0.0%
github.com/grafana/carbonapi/zipper/zipper.go 0.0%
github.com/grafana/carbonapi/zipper/zipper_pb3.go 0.0%
total 55.5%

Go lint report:

No issues found. 😎

Copy link
Collaborator

@carrieedwards carrieedwards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@npazosmendez npazosmendez changed the title Fix upstream merge conflicts Fix upstream merge conflicts [DO NOT MERGE] Aug 10, 2022
@npazosmendez npazosmendez changed the title Fix upstream merge conflicts [DO NOT MERGE] Fix upstream merge conflicts [DO NOT MERGE, just review] Aug 10, 2022
Copy link
Collaborator

@leizor leizor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants