Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatic Materialized View creation #53

Closed
wants to merge 41 commits into from
Closed

Automatic Materialized View creation #53

wants to merge 41 commits into from

Conversation

estebgonza
Copy link
Collaborator

  • Each aggregate pipeline that can be converted in a Materialized View will trigger a new Materialized View creation.

@estebgonza
resolve expression for__id and accumulators
b01cc14
@estebgonza
add ts-doc
a54538d
@estebgonza
add states to mv with automatic creation
1db04e7
@estebgonza
Merge branch 'main' into 44-materialized-views-retrieve-and-listen-da…
e0e768a 
…ta-from-mongodb
@estebgonza
integration: run mongo and maggregor on diff ports
f5c245b
@estebgonza
disable e2e atlas env var requirements
95d8735
@estebgonza
setup unit and integration tests
c75d944
@estebgonza
increase code coverage
c0a012f
@estebgonza
avg accumulator can be initialized
f82878e
@estebgonza
cached accumulators can be faulty
b58f339
@estebgonza
materialized views can be faulty
6d85b89
@estebgonza
increase code coverage
2738ee2
@estebgonza
implement queue system with nestjs/bullmq
cfbbd9f
@estebgonza
setup redis in e2e and benchmark
7da276f
@estebgonza
integration tests fixes
b0908e9
@estebgonza
increase hookTimeout for redis
f0dfb7d
@estebgonza
add integration test on creation queue
f230f40
@estebgonza
Merge branch 'main' into 44-materialized-views-retrieve-and-listen-da…
b26d832 
…ta-from-mongodb
@estebgonza
resolve pnpm-lock.yaml conflicts
38e13dd
@estebgonza
delete mv that are faulty
0baad1f
@estebgonza
update mv on changes
8c355f7
@estebgonza
add integration and unit tests
13d9f6c
@estebgonza
add redis configuration
b842eb3
@estebgonza estebgonza linked an issue May 26, 2023 that may be closed by this pull request
Automatic creation of Materialized Views #50
Open
@estebgonza estebgonza marked this pull request as draft May 26, 2023 10:36
@estebgonza estebgonza requested a review from NicoGGG May 26, 2023 10:36
@estebgonza estebgonza added the enhancement New feature or request label May 26, 2023
@estebgonza estebgonza added this to the Minimal valuable product milestone May 26, 2023
@estebgonza estebgonza changed the title Automatic Materialzied View creation Automatic Materialized View creation May 26, 2023
@estebgonza estebgonza force-pushed the 50-automatic-creation-of-materialized-views branch from 61f7317 to 1e48cc9 Compare May 26, 2023 18:36
@estebgonza estebgonza force-pushed the 50-automatic-creation-of-materialized-views branch from 1e48cc9 to 6ec8ec3 Compare May 26, 2023 18:48
@codecov
Copy link

codecov bot commented May 26, 2023 •
edited

Codecov Report

Merging #53 (700a185) into main (7177806) will increase coverage by 0.05%.
The diff coverage is 97.74%.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
+ Coverage   97.75%   97.81%   +0.05%     
==========================================
  Files          30       31       +1     
  Lines        2852     3061     +209     
  Branches      452      501      +49     
==========================================
+ Hits         2788     2994     +206     
- Misses         64       67       +3
Impacted Files Coverage Δ
...les/materialized-view/materialized-view.service.ts 95.04% <66.66%> (-2.31%) ⬇️
src/server/modules/cache-request/cache.service.ts 95.74% <85.71%> (+0.09%) ⬆️
src/core/materialized-view.ts 100.00% <100.00%> (+0.88%) ⬆️
src/core/pipeline/accumulators/cached.ts 98.37% <100.00%> (+0.03%) ⬆️
src/core/pipeline/accumulators/common.ts 94.50% <100.00%> (+0.52%) ⬆️
src/core/pipeline/accumulators/index.ts 93.93% <100.00%> (+0.06%) ⬆️
...s/materialized-view/materialized-view.converter.ts 100.00% <100.00%> (ø)
src/server/modules/request/request.service.ts 98.52% <100.00%> (ø)

@estebgonza estebgonza force-pushed the 50-automatic-creation-of-materialized-views branch from 47bc67e to 33d72e6 Compare May 26, 2023 19:32
@estebgonza estebgonza force-pushed the 50-automatic-creation-of-materialized-views branch from a1b1852 to 6cd10b5 Compare May 26, 2023 19:35
@estebgonza estebgonza force-pushed the 50-automatic-creation-of-materialized-views branch from e35fd3e to e0bf419 Compare May 27, 2023 20:15
@estebgonza
firts e2e mv auto creation
3206fb4
@estebgonza
fix redis setup
b5b5588
@estebgonza
test that reproduce the issue
0acd83e
@estebgonza
implement count accumulator for group expression
68678e8
@estebgonza
Merge branch 'main' into 55-materializedview-does-not-delete-correctl…
a35b300 
…y-documents
@estebgonza
refactorise and clean
67983cd
@estebgonza
refactorise and clean
a407543
@estebgonza
increase code coverage
a34eef5
@estebgonza
Merge branch 'main' into 50-automatic-creation-of-materialized-views
26d3233
@estebgonza
Merge branch '55-materializedview-does-not-delete-correctly-documents…
8231b93 
…' into 50-automatic-creation-of-materialized-views
@estebgonza
type the mv entries result and insert doc unit test
d68518a
@estebgonza
Merge branch '55-materializedview-does-not-delete-correctly-documents…
b001447 
…' into 50-automatic-creation-of-materialized-views
@estebgonza
add unit test when a new group must be added in the mv
ddf008e
@estebgonza
renaming MaterialziedViewEntryResult
700a185
@estebgonza estebgonza closed this Dec 11, 2023
@estebgonza estebgonza deleted the 50-automatic-creation-of-materialized-views branch December 11, 2023 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NicoGGG NicoGGG Awaiting requested review from NicoGGG

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
Minimal valuable product
Development

Successfully merging this pull request may close these issues.

Automatic creation of Materialized Views
1 participant
@estebgonza