Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregation operators refactor #2664

Merged
merged 30 commits into from May 25, 2023

Conversation

noisersup
Copy link
Member

@noisersup noisersup commented May 18, 2023

Description

Closes #2317.

Readiness checklist

  • I added/updated unit tests.
  • I added/updated integration/compatibility tests.
  • I added/updated comments and checked rendering.
  • I made spot refactorings.
  • I updated user documentation.
  • I ran task all, and it passed.
  • I ensured that PR title is good enough for the changelog.
  • (for maintainers only) I set Reviewers (@FerretDB/core), Labels, Project and project's Sprint fields.
  • I marked all done items in this checklist.

@noisersup noisersup added the not ready Issues that are not ready to be worked on; PRs that should skip CI label May 18, 2023
@codecov
Copy link

codecov bot commented May 18, 2023

Codecov Report

Merging #2664 (df3d8ab) into main (c80629d) will decrease coverage by 0.04%.
The diff coverage is 45.50%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2664      +/-   ##
==========================================
- Coverage   62.71%   62.67%   -0.04%     
==========================================
  Files         432      435       +3     
  Lines       22256    22378     +122     
==========================================
+ Hits        13958    14026      +68     
- Misses       7370     7425      +55     
+ Partials      928      927       -1     
Impacted Files Coverage Δ
...nal/handlers/common/aggregations/operators/type.go 0.00% <0.00%> (ø)
...regations/stages/projection/projection_iterator.go 88.88% <ø> (ø)
internal/handlers/commonerrors/error.go 80.00% <ø> (ø)
internal/handlers/commonerrors/errorcode_string.go 80.00% <ø> (ø)
...ernal/handlers/common/aggregations/stages/group.go 76.40% <15.15%> (-11.39%) ⬇️
...andlers/common/aggregations/operators/operators.go 47.05% <47.05%> (ø)
...ommon/aggregations/stages/projection/projection.go 82.20% <47.61%> (ø)
.../common/aggregations/operators/accumulators/sum.go 95.40% <57.14%> (ø)
...ggregations/operators/accumulators/accumulators.go 75.00% <75.00%> (ø)
...ommon/aggregations/operators/accumulators/count.go 95.45% <91.66%> (ø)
... and 1 more
Flag Coverage Δ
integration 56.06% <45.50%> (-0.01%) ⬇️
mongodb 5.02% <0.00%> (-0.03%) ⬇️
pg 55.97% <45.50%> (-0.01%) ⬇️
unit 24.89% <0.00%> (-0.17%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@noisersup noisersup removed the not ready Issues that are not ready to be worked on; PRs that should skip CI label May 22, 2023
@noisersup noisersup marked this pull request as ready for review May 22, 2023 18:20
@noisersup noisersup requested a review from a team as a code owner May 22, 2023 18:20
@noisersup noisersup enabled auto-merge (squash) May 22, 2023 18:21
@noisersup noisersup added the code/chore Code maintenance improvements label May 22, 2023
@noisersup noisersup requested review from a team and rumyantseva May 22, 2023 18:22
chilagrow
chilagrow previously approved these changes May 24, 2023
Copy link
Contributor

@chilagrow chilagrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, almost everything is ready.

Things to clarify are:

  • we shouldn't use go gen where it's not needed and we should delete previously generated files
  • iterators are still better than slices, so we don't need to keep everything in memory
  • switch - case with similar errors

Copy link
Contributor

@w84thesun w84thesun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good! Let's move the errors to a separate function as discussed on the call!

Copy link
Contributor

@w84thesun w84thesun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good! Let's move the errors to a separate function as discussed on the call!


// Error implements the error interface.
func (e *OperatorError) Error() string {
return fmt.Sprint(e.code)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using stringer and error names instead of error code will make things much simpler

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To simplify it as much as possible can't we just do:

var (
	// ErrWrongType indicates that operator field is not a document.
	ErrWrongType = fmt.Errorf("Invalid type of operator field (expected document)")

	// ErrEmptyField indicates that operator field does not specify any operator.
	ErrEmptyField = fmt.Errorf("The operator field is empty (expected document)")

	// ErrTooManyFields indicates that operator field specifes more than one operators.
	ErrTooManyFields = fmt.Errorf("The operator field specifies more than one operator")

	// ErrNotImplemented indicates that given operator is not implemented yet.
	ErrNotImplemented = fmt.Errorf("The operator is not implemented yet")
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but then exhaustive linter will not be able to help you

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't complain

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. It will never complain, even if you remove some cases

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, should I replace it with stringer? Also, we use similar logic in pgdb, regex, hanadb, and commonparams should it be replaced either?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on whatever the switch on those values should be exhaustive

@mergify
Copy link
Contributor

mergify bot commented May 24, 2023

@noisersup this pull request has merge conflicts.

@mergify mergify bot added the conflict PRs that have merge conflicts label May 24, 2023
@mergify mergify bot removed the conflict PRs that have merge conflicts label May 24, 2023
Copy link
Contributor

@w84thesun w84thesun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me that the questions we discussed are fixed! Great beginning to have a better structure for accumulators and operators!

@noisersup noisersup merged commit 9da7885 into FerretDB:main May 25, 2023
23 of 25 checks passed
@noisersup noisersup deleted the aggregation-operators-refactor branch May 25, 2023 09:19
@AlekSi AlekSi added this to the v1.3.0 milestone Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code/chore Code maintenance improvements
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Implement operator interface in aggregation
5 participants