Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(core-transactions): bootstrap transactions in batches #2997

Merged
merged 3 commits into from Oct 3, 2019

Conversation

@supaiku0
Copy link
Contributor

commented Oct 1, 2019

Summary

Prevents memory spikes during bootstrap phase by loading transactions of
the same type in batches instead of all at once into memory.

Checklist

  • Documentation (if necessary)
  • Tests (if necessary)
  • Ready to be merged
Prevents memory spikes during bootstrap phase by loading transactions of
the same type in batches instead of all at once into memory during
bootstrap.
@codecov

This comment has been minimized.

Copy link

commented Oct 1, 2019

Codecov Report

Merging #2997 into develop will decrease coverage by 0.18%.
The diff coverage is 10.97%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2997      +/-   ##
===========================================
- Coverage    66.05%   65.86%   -0.19%     
===========================================
  Files          420      421       +1     
  Lines        10035    10082      +47     
  Branches       509      510       +1     
===========================================
+ Hits          6629     6641      +12     
- Misses        3364     3399      +35     
  Partials        42       42
Impacted Files Coverage Δ
...database-postgres/src/repositories/transactions.ts 24.07% <0%> (-0.46%) ⬇️
...c/repositories/transactions-business-repository.ts 87.09% <0%> (-1.43%) ⬇️
...nsactions/src/handlers/bridgechain-registration.ts 70.73% <10%> (-1.77%) ⬇️
...ansactions/src/handlers/bridgechain-resignation.ts 50% <11.11%> (-1.22%) ⬇️
...es/core-transactions/src/handlers/multi-payment.ts 77.14% <12.5%> (-1.65%) ⬇️
...trate-transactions/src/handlers/business-update.ts 26.66% <13.33%> (-3.34%) ⬇️
packages/core-transactions/src/handlers/ipfs.ts 74.07% <14.28%> (-1.93%) ⬇️
...transactions/src/handlers/business-registration.ts 68.75% <16.66%> (-2.22%) ⬇️
...-transactions/src/handlers/business-resignation.ts 56.25% <16.66%> (-1.82%) ⬇️
...transactions/src/handlers/delegate-registration.ts 43.85% <16.66%> (+0.22%) ⬆️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4ee4c0...88ae34a. Read the comment docs.

@codecov-io

This comment has been minimized.

Copy link

commented Oct 3, 2019

Codecov Report

❗️ No coverage uploaded for pull request base (develop@4953e48). Click here to learn what that means.
The diff coverage is 10.97%.

Impacted file tree graph

@@            Coverage Diff             @@
##             develop    #2997   +/-   ##
==========================================
  Coverage           ?   65.86%           
==========================================
  Files              ?      421           
  Lines              ?    10082           
  Branches           ?      510           
==========================================
  Hits               ?     6641           
  Misses             ?     3399           
  Partials           ?       42
Impacted Files Coverage Δ
...database-postgres/src/repositories/transactions.ts 24.07% <0%> (ø)
...c/repositories/transactions-business-repository.ts 87.09% <0%> (ø)
...nsactions/src/handlers/bridgechain-registration.ts 70.73% <10%> (ø)
...ansactions/src/handlers/bridgechain-resignation.ts 50% <11.11%> (ø)
...es/core-transactions/src/handlers/multi-payment.ts 77.14% <12.5%> (ø)
...trate-transactions/src/handlers/business-update.ts 26.66% <13.33%> (ø)
packages/core-transactions/src/handlers/ipfs.ts 74.07% <14.28%> (ø)
...transactions/src/handlers/business-registration.ts 68.75% <16.66%> (ø)
...-transactions/src/handlers/business-resignation.ts 56.25% <16.66%> (ø)
...transactions/src/handlers/delegate-registration.ts 43.85% <16.66%> (ø)
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4953e48...6aee57f. Read the comment docs.

@faustbrian faustbrian merged commit 27132f0 into develop Oct 3, 2019
81 checks passed
81 checks passed
crypto (10.x)
Details
bridgechain-registration (10.x)
Details
unit (10.x)
Details
crypto (12.x)
Details
bridgechain-registration (12.x)
Details
unit (12.x)
Details
bridgechain-resignation (10.x)
Details
integration (10.x)
Details
bridgechain-resignation (12.x)
Details
integration (12.x)
Details
bridgechain-update (10.x)
Details
e2e (10.x)
Details
bridgechain-update (12.x)
Details
e2e (12.x)
Details
business-registration (10.x)
Details
business-registration (12.x)
Details
business-resignation (10.x)
Details
business-resignation (12.x)
Details
business-update (10.x)
Details
business-update (12.x)
Details
delegate-registration (10.x)
Details
delegate-registration (12.x)
Details
delegate-resignation (10.x)
Details
delegate-resignation (12.x)
Details
htlc-claim (10.x)
Details
htlc-claim (12.x)
Details
htlc-lock (10.x)
Details
htlc-lock (12.x)
Details
htlc-refund (10.x)
Details
htlc-refund (12.x)
Details
ipfs (10.x)
Details
ipfs (12.x)
Details
multi-payment (10.x)
Details
multi-payment (12.x)
Details
multi-signature-registration (10.x)
Details
multi-signature-registration (12.x)
Details
second-signature-registration (10.x)
Details
second-signature-registration (12.x)
Details
transfer (10.x)
Details
transfer (12.x)
Details
vote (10.x)
Details
vote (12.x)
Details
LGTM analysis: JavaScript No new or fixed alerts
Details
codeclimate Approved by Brian Faust.
Details
security/snyk - __tests__/e2e/package.json (ArkEcosystem) No manifest changes detected
security/snyk - package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-api/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-blockchain/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-container/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-database-postgres/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-database/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-elasticsearch/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-error-tracker-airbrake/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-error-tracker-bugsnag/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-error-tracker-raygun/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-error-tracker-rollbar/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-error-tracker-sentry/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-event-emitter/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-exchange-json-rpc/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-explorer/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-forger/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-http-utils/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-interfaces/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-jest-matchers/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-logger-pino/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-logger-signale/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-logger-winston/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-logger/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-new-relic/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-p2p/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-snapshots/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-state/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-tester-cli/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-transaction-pool/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-transactions/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-utils/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-vote-report/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-wallet-api/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core-webhooks/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/core/package.json (ArkEcosystem) No manifest changes detected
security/snyk - packages/crypto/package.json (ArkEcosystem) No manifest changes detected
@ArkEcosystemBot ArkEcosystemBot deleted the refactor/core-transactions/bootstrap branch Oct 3, 2019
vasild added a commit that referenced this pull request Oct 4, 2019
…ts-nonce

* ArkEcosystem/core/develop:
  refactor: remove `vendorFieldHex` (#3014)
  ci: run push events on master and develop, rest on PRs
  perf(crypto): memoize base58 de/encoding (#3015)
  refactor(core-transactions): bootstrap transactions in batches (#2997)
  perf(crypto): replace bignumber.js with native BigInt (#3010)
  refactor: add `round.missed` event (#3011)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.