Rework integration test setup for validator #2225

chilagrow · 2023-03-17T05:39:23Z

Description

Closes #1568.

So actually, validator was already only applied for target by if condition.
This PR uses backend in test setup and shared data, and removed handler references.

Readiness checklist

I added/updated unit tests.
I added/updated integration/compatibility tests.
I added/updated comments and checked rendering.
I made spot refactorings.
I updated user documentation.
I ran task all, and it passed.
I ensured that PR title is good enough for the changelog.
(for maintainers only) I set Reviewers (@FerretDB/core), Assignee, Labels, Project and project's Sprint fields.
I marked all done items in this checklist.

codecov · 2023-03-17T07:55:29Z

Codecov Report

Merging #2225 (aa10330) into main (5705db7) will decrease coverage by 0.04%.
The diff coverage is 88.23%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2225      +/-   ##
==========================================
- Coverage   70.91%   70.87%   -0.04%     
==========================================
  Files         371      371              
  Lines       17324    17308      -16     
==========================================
- Hits        12285    12267      -18     
  Misses       4014     4014              
- Partials     1025     1027       +2

Impacted Files	Coverage Δ
integration/helpers.go	`71.87% <ø> (ø)`
integration/setup/listener.go	`85.12% <ø> (+0.39%)`	⬆️
integration/setup/test_helpers.go	`100.00% <ø> (ø)`
integration/shareddata/scalars.go	`100.00% <ø> (ø)`
integration/setup/setup.go	`79.56% <20.00%> (-0.59%)`	⬇️
integration/setup/setup_compat.go	`88.23% <100.00%> (+0.63%)`	⬆️
integration/shareddata/shareddata.go	`88.65% <100.00%> (+11.18%)`	⬆️

... and 3 files with indirect coverage changes

Flag	Coverage Δ
integration	`66.72% <88.23%> (+<0.01%)`	⬆️
mongodb	`5.14% <8.82%> (-0.02%)`	⬇️
pg	`54.85% <38.23%> (-0.01%)`	⬇️
tigris	`46.64% <88.23%> (+0.07%)`	⬆️
unit	`26.81% <ø> (-0.22%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

w84thesun

Nice job! It makes tests a lot more consistent!

AlekSi

Let's also try to remove documentValidationFailureCode by marking data provides as incompatible or by updating schema/validators

This reverts commit 624b5d6.

chilagrow · 2023-03-22T04:33:27Z

Let's also try to remove documentValidationFailureCode by marking data provides as incompatible or by updating schema/validators

@AlekSi this requires quite a bit more work.
For example, an operator that update the value like bson.D{{"$set", bson.D{{"v", int32(42)}}}} would pass on Int32s provider but fail on all other providers, because tigris schema won't be the right one for other ones.
We need to think of a setup to only run tigris test on specific provider. It kind of feels like we may need to set tigris schema per test case.

I suggest handling this in separate PR. #2253

AlekSi · 2023-03-22T06:29:51Z

@chilagrow sounds good, let's add a short comment and TODO URL

@rumyantseva

v0.9.4 https://github.com/FerretDB/FerretDB/releases/tag/v0.9.4 2023-03-27 * Support dot notation in sorting by @rumyantseva in FerretDB#2156 * Add support for `$each` modifier to `$addToSet` array update operator by @w84thesun in FerretDB#2187 * Add support for `$each` modifier to `$push` array update operator by @w84thesun in FerretDB#2202 * Add support for `$pull` array update operator by @w84thesun in FerretDB#2222 * Fix `hana` and `tigris` handlers initialization by @AlekSi in FerretDB#2234 * Fix long overflow detection on `$mul` by @noisersup in FerretDB#2232 * Fix pushdown for large double values by @noisersup in FerretDB#2163 * Support `skip` argument for `find` and `count` by @rumyantseva in FerretDB#2193 * Support query pushdown for `$match` aggregation stage by @chilagrow in FerretDB#2199 * Rework integration test setup for validator by @chilagrow in FerretDB#2225 * Change collection names mangling for PostgreSQL by @chilagrow in FerretDB#2239 * Implement `nameOnly` for `listCollections` command by @kropidlowsky in FerretDB#2247 * Extract dollar path field expression to types package by @chilagrow in FerretDB#2263 * Prevent confusing messages for embedded FerretDB by @AlekSi in FerretDB#2209 * Add new FerretDB v0.9.3 release blogpost by @Fashander in FerretDB#2201 * Add documentation for `all-in-one` Docker image by @AlekSi in FerretDB#2215 * Tweak Docker docs a bit by @AlekSi in FerretDB#2228 * Add section about field update operators by @Fashander in FerretDB#2180 * Update telemetry information by @Fashander in FerretDB#2243 * Add basic security policy by @AlekSi in FerretDB#2242 * Add new blog post - MongoDB vs PostgreSQL by @Fashander in FerretDB#2146 * Document array update operator by @Fashander in FerretDB#2237 * Merge and speedup packages building by @AlekSi in FerretDB#2195 * Provide "all-in-one" Docker image by @AlekSi in FerretDB#2183 * Improve logging of retries by @AlekSi in FerretDB#2192 * Split `TestQueryCompatRunner` back into separate tests by @chilagrow in FerretDB#2189 * Fix telemetry tag and improve unimplemented error in aggregation by @chilagrow in FerretDB#2198 * Improve build cache usage by @AlekSi in FerretDB#2197 * Add CI configuration for Hana by @AlekSi in FerretDB#2194 * Use `runit` for all-in-one Docker image by @AlekSi in FerretDB#2203 * Pass `struct`s by pointers by @AlekSi in FerretDB#2221 * Skip a `$mod` test for arm64 by @rumyantseva in FerretDB#2223 * Fix `env-data` Taskfile by @chilagrow in FerretDB#2211 * Make `listIndexes` use data from the database by @rumyantseva in FerretDB#2214 * Tweak dependabot config by @AlekSi in FerretDB#2233 * Filter documents with iterators by @AlekSi in FerretDB#2245 * Add a few query tests by @AlekSi in FerretDB#2260 * Fix traffic recording by @AlekSi in FerretDB#2267 * Cleanup unimplemented and ignored parameters by @AlekSi in FerretDB#2258 * Bump deps by @AlekSi in FerretDB#2264 * Collection deletion and metadata refactoring by @rumyantseva in FerretDB#2224 * Update diff.md for database name differences by @chilagrow in FerretDB#2274 * Remove indexes from `DocumentsIterator` by @AlekSi in FerretDB#2278 * Bump Tigris to 1.0.0-beta.51 by @AlekSi in FerretDB#2277 * @kropidlowsky made their first contribution in FerretDB#2247 All closed issues and pull requests: https://github.com/FerretDB/FerretDB/milestone/38?closed=1 All commits: FerretDB/FerretDB@v0.9.3...v0.9.4

@rumyantseva

v0.9.4 https://github.com/FerretDB/FerretDB/releases/tag/v0.9.4 2023-03-27 * Support dot notation in sorting by @rumyantseva in FerretDB#2156 * Add support for `$each` modifier to `$addToSet` array update operator by @w84thesun in FerretDB#2187 * Add support for `$each` modifier to `$push` array update operator by @w84thesun in FerretDB#2202 * Add support for `$pull` array update operator by @w84thesun in FerretDB#2222 * Fix `hana` and `tigris` handlers initialization by @AlekSi in FerretDB#2234 * Fix long overflow detection on `$mul` by @noisersup in FerretDB#2232 * Fix pushdown for large double values by @noisersup in FerretDB#2163 * Support `skip` argument for `find` and `count` by @rumyantseva in FerretDB#2193 * Support query pushdown for `$match` aggregation stage by @chilagrow in FerretDB#2199 * Rework integration test setup for validator by @chilagrow in FerretDB#2225 * Change collection names mangling for PostgreSQL by @chilagrow in FerretDB#2239 * Implement `nameOnly` for `listCollections` command by @kropidlowsky in FerretDB#2247 * Extract dollar path field expression to types package by @chilagrow in FerretDB#2263 * Prevent confusing messages for embedded FerretDB by @AlekSi in FerretDB#2209 * Add new FerretDB v0.9.3 release blogpost by @Fashander in FerretDB#2201 * Add documentation for `all-in-one` Docker image by @AlekSi in FerretDB#2215 * Tweak Docker docs a bit by @AlekSi in FerretDB#2228 * Add section about field update operators by @Fashander in FerretDB#2180 * Update telemetry information by @Fashander in FerretDB#2243 * Add basic security policy by @AlekSi in FerretDB#2242 * Add new blog post - MongoDB vs PostgreSQL by @Fashander in FerretDB#2146 * Document array update operator by @Fashander in FerretDB#2237 * Merge and speedup packages building by @AlekSi in FerretDB#2195 * Provide "all-in-one" Docker image by @AlekSi in FerretDB#2183 * Improve logging of retries by @AlekSi in FerretDB#2192 * Split `TestQueryCompatRunner` back into separate tests by @chilagrow in FerretDB#2189 * Fix telemetry tag and improve unimplemented error in aggregation by @chilagrow in FerretDB#2198 * Improve build cache usage by @AlekSi in FerretDB#2197 * Add CI configuration for Hana by @AlekSi in FerretDB#2194 * Use `runit` for all-in-one Docker image by @AlekSi in FerretDB#2203 * Pass `struct`s by pointers by @AlekSi in FerretDB#2221 * Skip a `$mod` test for arm64 by @rumyantseva in FerretDB#2223 * Fix `env-data` Taskfile by @chilagrow in FerretDB#2211 * Make `listIndexes` use data from the database by @rumyantseva in FerretDB#2214 * Tweak dependabot config by @AlekSi in FerretDB#2233 * Filter documents with iterators by @AlekSi in FerretDB#2245 * Add a few query tests by @AlekSi in FerretDB#2260 * Fix traffic recording by @AlekSi in FerretDB#2267 * Cleanup unimplemented and ignored parameters by @AlekSi in FerretDB#2258 * Bump deps by @AlekSi in FerretDB#2264 * Collection deletion and metadata refactoring by @rumyantseva in FerretDB#2224 * Update diff.md for database name differences by @chilagrow in FerretDB#2274 * Remove indexes from `DocumentsIterator` by @AlekSi in FerretDB#2278 * Bump Tigris to 1.0.0-beta.51 by @AlekSi in FerretDB#2277 * @kropidlowsky made their first contribution in FerretDB#2247 All closed issues and pull requests: https://github.com/FerretDB/FerretDB/milestone/38?closed=1 All commits: FerretDB/FerretDB@v0.9.3...v0.9.4

@rumyantseva

v0.9.4 https://github.com/FerretDB/FerretDB/releases/tag/v0.9.4 2023-03-27 * Support dot notation in sorting by @rumyantseva in FerretDB#2156 * Add support for `$each` modifier to `$addToSet` array update operator by @w84thesun in FerretDB#2187 * Add support for `$each` modifier to `$push` array update operator by @w84thesun in FerretDB#2202 * Add support for `$pull` array update operator by @w84thesun in FerretDB#2222 * Fix `hana` and `tigris` handlers initialization by @AlekSi in FerretDB#2234 * Fix long overflow detection on `$mul` by @noisersup in FerretDB#2232 * Fix pushdown for large double values by @noisersup in FerretDB#2163 * Support `skip` argument for `find` and `count` by @rumyantseva in FerretDB#2193 * Support query pushdown for `$match` aggregation stage by @chilagrow in FerretDB#2199 * Rework integration test setup for validator by @chilagrow in FerretDB#2225 * Change collection names mangling for PostgreSQL by @chilagrow in FerretDB#2239 * Implement `nameOnly` for `listCollections` command by @kropidlowsky in FerretDB#2247 * Extract dollar path field expression to types package by @chilagrow in FerretDB#2263 * Prevent confusing messages for embedded FerretDB by @AlekSi in FerretDB#2209 * Add new FerretDB v0.9.3 release blogpost by @Fashander in FerretDB#2201 * Add documentation for `all-in-one` Docker image by @AlekSi in FerretDB#2215 * Tweak Docker docs a bit by @AlekSi in FerretDB#2228 * Add section about field update operators by @Fashander in FerretDB#2180 * Update telemetry information by @Fashander in FerretDB#2243 * Add basic security policy by @AlekSi in FerretDB#2242 * Add new blog post - MongoDB vs PostgreSQL by @Fashander in FerretDB#2146 * Document array update operator by @Fashander in FerretDB#2237 * Merge and speedup packages building by @AlekSi in FerretDB#2195 * Provide "all-in-one" Docker image by @AlekSi in FerretDB#2183 * Improve logging of retries by @AlekSi in FerretDB#2192 * Split `TestQueryCompatRunner` back into separate tests by @chilagrow in FerretDB#2189 * Fix telemetry tag and improve unimplemented error in aggregation by @chilagrow in FerretDB#2198 * Improve build cache usage by @AlekSi in FerretDB#2197 * Add CI configuration for Hana by @AlekSi in FerretDB#2194 * Use `runit` for all-in-one Docker image by @AlekSi in FerretDB#2203 * Pass `struct`s by pointers by @AlekSi in FerretDB#2221 * Skip a `$mod` test for arm64 by @rumyantseva in FerretDB#2223 * Fix `env-data` Taskfile by @chilagrow in FerretDB#2211 * Make `listIndexes` use data from the database by @rumyantseva in FerretDB#2214 * Tweak dependabot config by @AlekSi in FerretDB#2233 * Filter documents with iterators by @AlekSi in FerretDB#2245 * Add a few query tests by @AlekSi in FerretDB#2260 * Fix traffic recording by @AlekSi in FerretDB#2267 * Cleanup unimplemented and ignored parameters by @AlekSi in FerretDB#2258 * Bump deps by @AlekSi in FerretDB#2264 * Collection deletion and metadata refactoring by @rumyantseva in FerretDB#2224 * Update diff.md for database name differences by @chilagrow in FerretDB#2274 * Remove indexes from `DocumentsIterator` by @AlekSi in FerretDB#2278 * Bump Tigris to 1.0.0-beta.51 by @AlekSi in FerretDB#2277 * @kropidlowsky made their first contribution in FerretDB#2247 All closed issues and pull requests: https://github.com/FerretDB/FerretDB/milestone/38?closed=1 All commits: FerretDB/FerretDB@v0.9.3...v0.9.4

@rumyantseva

v0.9.4 https://github.com/FerretDB/FerretDB/releases/tag/v0.9.4 2023-03-27 * Support dot notation in sorting by @rumyantseva in FerretDB#2156 * Add support for `$each` modifier to `$addToSet` array update operator by @w84thesun in FerretDB#2187 * Add support for `$each` modifier to `$push` array update operator by @w84thesun in FerretDB#2202 * Add support for `$pull` array update operator by @w84thesun in FerretDB#2222 * Fix `hana` and `tigris` handlers initialization by @AlekSi in FerretDB#2234 * Fix long overflow detection on `$mul` by @noisersup in FerretDB#2232 * Fix pushdown for large double values by @noisersup in FerretDB#2163 * Support `skip` argument for `find` and `count` by @rumyantseva in FerretDB#2193 * Support query pushdown for `$match` aggregation stage by @chilagrow in FerretDB#2199 * Rework integration test setup for validator by @chilagrow in FerretDB#2225 * Change collection names mangling for PostgreSQL by @chilagrow in FerretDB#2239 * Implement `nameOnly` for `listCollections` command by @kropidlowsky in FerretDB#2247 * Extract dollar path field expression to types package by @chilagrow in FerretDB#2263 * Prevent confusing messages for embedded FerretDB by @AlekSi in FerretDB#2209 * Add new FerretDB v0.9.3 release blogpost by @Fashander in FerretDB#2201 * Add documentation for `all-in-one` Docker image by @AlekSi in FerretDB#2215 * Tweak Docker docs a bit by @AlekSi in FerretDB#2228 * Add section about field update operators by @Fashander in FerretDB#2180 * Update telemetry information by @Fashander in FerretDB#2243 * Add basic security policy by @AlekSi in FerretDB#2242 * Add new blog post - MongoDB vs PostgreSQL by @Fashander in FerretDB#2146 * Document array update operator by @Fashander in FerretDB#2237 * Merge and speedup packages building by @AlekSi in FerretDB#2195 * Provide "all-in-one" Docker image by @AlekSi in FerretDB#2183 * Improve logging of retries by @AlekSi in FerretDB#2192 * Split `TestQueryCompatRunner` back into separate tests by @chilagrow in FerretDB#2189 * Fix telemetry tag and improve unimplemented error in aggregation by @chilagrow in FerretDB#2198 * Improve build cache usage by @AlekSi in FerretDB#2197 * Add CI configuration for Hana by @AlekSi in FerretDB#2194 * Use `runit` for all-in-one Docker image by @AlekSi in FerretDB#2203 * Pass `struct`s by pointers by @AlekSi in FerretDB#2221 * Skip a `$mod` test for arm64 by @rumyantseva in FerretDB#2223 * Fix `env-data` Taskfile by @chilagrow in FerretDB#2211 * Make `listIndexes` use data from the database by @rumyantseva in FerretDB#2214 * Tweak dependabot config by @AlekSi in FerretDB#2233 * Filter documents with iterators by @AlekSi in FerretDB#2245 * Add a few query tests by @AlekSi in FerretDB#2260 * Fix traffic recording by @AlekSi in FerretDB#2267 * Cleanup unimplemented and ignored parameters by @AlekSi in FerretDB#2258 * Bump deps by @AlekSi in FerretDB#2264 * Collection deletion and metadata refactoring by @rumyantseva in FerretDB#2224 * Update diff.md for database name differences by @chilagrow in FerretDB#2274 * Remove indexes from `DocumentsIterator` by @AlekSi in FerretDB#2278 * Bump Tigris to 1.0.0-beta.51 by @AlekSi in FerretDB#2277 * @kropidlowsky made their first contribution in FerretDB#2247 All closed issues and pull requests: https://github.com/FerretDB/FerretDB/milestone/38?closed=1 All commits: FerretDB/FerretDB@v0.9.3...v0.9.4

chilagrow added 2 commits March 16, 2023 17:04

provider checks compatible backend

77cd47b

compat tests remove references of handlers, they are backends

9602663

chilagrow added the code/enhancement Some user-visible feature could work better label Mar 17, 2023

chilagrow self-assigned this Mar 17, 2023

chilagrow added 6 commits March 17, 2023 14:39

Merge branch 'main' into integration-test-validator

72a2dcf

update comment

49f8b82

db name has no dash

cf7adcc

fix failing test due too db name too long

36999d2

remove duplicated underscore

23c18d7

comment

dbf460f

chilagrow changed the title ~~Compat test applies validator on target backend~~ Rework integration test setup for validator Mar 17, 2023

chilagrow requested review from a team, w84thesun, rumyantseva and noisersup March 17, 2023 07:58

chilagrow marked this pull request as ready for review March 17, 2023 07:58

chilagrow requested a review from a team as a code owner March 17, 2023 07:58

chilagrow requested a review from AlekSi March 17, 2023 07:58

Merge branch 'main' into integration-test-validator

3e7ce79

chilagrow enabled auto-merge (squash) March 17, 2023 07:59

mergify bot and others added 3 commits March 17, 2023 09:58

Merge branch 'main' into integration-test-validator

1ecd3a1

merge

d164574

Merge branch 'main' into integration-test-validator

58a2a35

w84thesun previously approved these changes Mar 17, 2023

View reviewed changes

mergify bot added 3 commits March 17, 2023 16:43

Merge branch 'main' into integration-test-validator

3050921

Merge branch 'main' into integration-test-validator

b551c3c

Merge branch 'main' into integration-test-validator

7d64db6

rumyantseva previously approved these changes Mar 20, 2023

View reviewed changes

AlekSi requested changes Mar 20, 2023

View reviewed changes

noisersup previously approved these changes Mar 20, 2023

View reviewed changes

remove validation failure code error

624b5d6

chilagrow dismissed stale reviews from noisersup, rumyantseva, and w84thesun via 624b5d6 March 22, 2023 02:53

chilagrow added 2 commits March 22, 2023 11:53

Merge branch 'main' into integration-test-validator

f813534

Revert "remove validation failure code error"

3e7a4f1

This reverts commit 624b5d6.

chilagrow added 4 commits March 22, 2023 16:11

add todo for removing validator error for tigris in compat test

64e1c4c

add comment

6ca9348

refactor

12a89c3

fix missing backend

aa10330

chilagrow requested review from AlekSi, w84thesun, rumyantseva and noisersup March 22, 2023 07:36

rumyantseva approved these changes Mar 22, 2023

View reviewed changes

AlekSi approved these changes Mar 22, 2023

View reviewed changes

chilagrow merged commit 07134cf into FerretDB:main Mar 22, 2023

AlekSi added this to the v0.9.4 milestone Mar 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework integration test setup for validator #2225

Rework integration test setup for validator #2225

chilagrow commented Mar 17, 2023 •

edited

Loading

codecov bot commented Mar 17, 2023 •

edited

Loading

w84thesun left a comment

AlekSi left a comment •

edited

Loading

chilagrow commented Mar 22, 2023

AlekSi commented Mar 22, 2023

Rework integration test setup for validator #2225

Rework integration test setup for validator #2225

Conversation

chilagrow commented Mar 17, 2023 • edited Loading

Description

Readiness checklist

codecov bot commented Mar 17, 2023 • edited Loading

Codecov Report

w84thesun left a comment

Choose a reason for hiding this comment

AlekSi left a comment • edited Loading

Choose a reason for hiding this comment

chilagrow commented Mar 22, 2023

AlekSi commented Mar 22, 2023

chilagrow commented Mar 17, 2023 •

edited

Loading

codecov bot commented Mar 17, 2023 •

edited

Loading

AlekSi left a comment •

edited

Loading