Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor data migration to lazily load results #719

Merged
merged 2 commits into from
Apr 29, 2019
Merged

Conversation

prd-fox
Copy link
Contributor

@prd-fox prd-fox commented Apr 29, 2019

Refactor data migration to lazily load results as to not blow memory.
The loaders now return one result at a time.
Removed dud test files and service definitions that were not used.

Fixes #705

The loaders now return one result at a time.
Removed dud test files and service definitions that were not used.
@prd-fox prd-fox changed the title Refactor data migration to lazily load results as to not blow memory. Refactor data migration to lazily load results Apr 29, 2019
@codecov-io
Copy link

codecov-io commented Apr 29, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@7236fe9). Click here to learn what that means.
The diff coverage is 99.41%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #719   +/-   ##
=========================================
  Coverage          ?   99.48%           
  Complexity        ?     1894           
=========================================
  Files             ?      286           
  Lines             ?     5619           
  Branches          ?      285           
=========================================
  Hits              ?     5590           
  Misses            ?        5           
  Partials          ?       24
Impacted Files Coverage Δ Complexity Δ
...om/quorum/tessera/data/migration/JdbcCallback.java 100% <ø> (ø) 1 <0> (?)
...m/tessera/data/migration/StoreLoaderException.java 100% <ø> (ø) 1 <0> (?)
...om/quorum/tessera/data/migration/SqliteLoader.java 100% <100%> (ø) 4 <3> (?)
...quorum/tessera/data/migration/CmdLineExecutor.java 100% <100%> (ø) 6 <5> (?)
...um/tessera/data/migration/DataExporterFactory.java 100% <100%> (ø) 3 <0> (?)
...uorum/tessera/data/migration/JdbcDataExporter.java 100% <100%> (ø) 4 <1> (?)
.../quorum/tessera/data/migration/H2DataExporter.java 100% <100%> (ø) 4 <2> (?)
...a/com/quorum/tessera/data/migration/DataEntry.java 100% <100%> (ø) 3 <3> (?)
...rum/tessera/data/migration/DirectoryStoreFile.java 100% <100%> (ø) 5 <4> (?)
...rum/tessera/data/migration/SqliteDataExporter.java 100% <100%> (ø) 6 <1> (?)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7236fe9...9d9aefe. Read the comment docs.

@melowe melowe merged commit de744b2 into master Apr 29, 2019
@prd-fox prd-fox added 0.10.0 bug Something isn't working labels Apr 29, 2019
@Krish1979 Krish1979 added 0.9.3 and removed 0.10.0 labels Jul 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.9.3 bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to migrate data from constellation when payloads content is large
4 participants