Separate business-logic from data-layer logic in core-database/postgres by paroxysm · Pull Request #2055 · ArkEcosystem/core

paroxysm · 2019-02-03T20:21:39Z

Proposed changes

Resolves #2006

The PR introduces a hierarchical change to the core-database/postgres.

I've encapsulated the business logic, previously in ConnectionInterface and PostgresConnection, into a new class DatabaseService, implements IDatabaseService, which references a database specific implementation of the IDatabaseConnection interface via constructor injection.

IDatabaseConnection supplies the repos that DatabaseService uses in certain methods.

The plugin providing a IDatabaseConnection concrete class is responsible for instantiating a DatabaseService and supply a IDatabaseConnection concrete class, among with other dependencies such as IWalletManager

In theory, the plugin can also provide its own implementation of IWalletManager and such..

The implication is that we're now exposing an IDatabaseService implementation to be used by the other modules. Most of these modules don't care about the underlying data-layer and shouldn't.
The exception would be core-api/core-graphql/core-snapshots which are(oddly) defining repositories that are accessing postgres specific internals(I use 'any' type-casting to get past compilation errors but this tech-debt that should be addressed sooner than later). These external repos need to be relocated into IDatabaseConnection repos and w/e additional logic they were doing be merged.

It shouldn't be too hard to convert the search/query code these external repositories are using into a generic implementation on the database specific repos.

I've tested that my relay runs and syncs as usual. I haven't tested some other affected modules such as core-graphql/snapshot outside their tests

The next step would be to start improving the DatabaseService tests, now that we can stub easier.

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Refactoring (improve a current implementation without adding a new feature or fixing a bug)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Build (changes that affect the build system)
Docs (documentation only changes)
Test (adding missing tests or fixing existing tests)
Other... Please describe:

Checklist

I have read the CONTRIBUTING documentation
Lint and unit tests pass locally with my changes
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

…s instead. Use type-casts to mask compile errors due accessing of private/internal details publicy.

…Then use said typed repos in methods

…r-core-database # Conflicts: # packages/core-transaction-pool/__tests__/connection.test.ts

…r-core-database # Conflicts: # packages/core-database-postgres/src/spv.ts

…r-core-database

fix: missing crucial line of code in 'DatabaseService.getForgedTransactionIds'

ghost · 2019-02-03T20:21:42Z

@paroxysm Thanks for submitting this pull request, a maintainer will get back to you shortly!

ghost · 2019-02-03T20:21:42Z

@faustbrian @supaiku0 @air1one - please review this in the next few days. Be sure to explicitly select labels so I know what's going on.

If no reviewer appears after a week, a reminder will be sent out.

ghost · 2019-02-03T20:21:44Z

@paroxysm Your pull request doesn't follow our contribution guidelines. Please review and correct it.

…r-core-database

faustbrian · 2019-02-04T11:13:36Z

        const manager = new SnapshotManager(options);

-        return manager.make(container.resolvePlugin<PostgresConnection>("database"));
+        const connection = container.resolvePlugin<Database.IDatabaseService>("database").connection as any;


This causes some resolution issues. Try yarn dump:testnet --config ../core/src/config/testnet through the CLI to reproduce it.

TypeError: Cannot read property 'connection' of null at Object.register (./packages/core-snapshots/dist/plugin.js:15:63)

@faustbrian I've solved the issue.

paroxysm · 2019-02-04T14:20:01Z

I'll take a look after work 👍🏿
EDIT:

I've introduced a factory function for instantiating the DatabaseService class. I didn't like core-database extenders having to know of and being able to override the business repos.

By using the factory function, the extenders only need to know of an IWalletManager and IDatabaseConnection. It makes sense that the former can be overridden for a different implementation that can also be database specific(Right now it uses an in-memory model, but perhaps a future one will persist the wallets in the database or use cache employing a db-backing)

Also, the factory function allows us to break-up parts of the DatabaseService implementation into separate chunks(the more complicated methods). Since they all come together via the constructor, extenders of the core-database would also need to know of those chunks too, which isn't is intended.

…ng up w/o a db connection) refactor: export a database-service factory for instantiating it. This prevents db flavor implementors from overriding the business repos which have expected implementations already.

…r-core-database

ghost · 2019-02-05T02:55:28Z

@paroxysm The ci/circleci: test-node11-0 job is failing as of 0a091e13265aa380307cf1176da82322828ed654. Please review the logs for more information.

Once you've pushed the fixes, the build will automatically re-run. Thanks!

ghost · 2019-02-05T06:26:33Z

@paroxysm Your pull request has been merged and marked as tier 0. It needs a specialized, in-depth review before we can decide on its value.

faustbrian · 2019-02-05T07:19:50Z

@paroxysm do you want to take care of extracting the repositories out of core-api/graphql?

paroxysm · 2019-02-05T13:21:08Z

Yea, I'll handle that

#2055)

…peer-state * ArkEcosystem/core/develop: refactor(core-database): separate business-logic from data-layer logic (#2055)

paroxysm added 15 commits January 27, 2019 23:04

refactor: refactor core-database to use interfaces, first pass.

e7fa156

refactor: remove dep on core-database-postgres and use interface type…

54286c7

…s instead. Use type-casts to mask compile errors due accessing of private/internal details publicy.

refactor: have postgres repos implement their respective interfaces. …

4bfac6d

…Then use said typed repos in methods

refactor: use event-emitter to fire some database life-cycle events

2df6216

Merge remote-tracking branch 'arkecosystem/develop' into feat/refacto…

d79db8c

…r-core-database # Conflicts: # packages/core-transaction-pool/__tests__/connection.test.ts

Merge remote-tracking branch 'arkecosystem/develop' into feat/refacto…

dccea07

…r-core-database # Conflicts: # packages/core-database-postgres/src/spv.ts

fix: wallet-manager tests

01fdf5e

fix: delegates and wallets business repo tests.

3241b51

Merge remote-tracking branch 'arkecosystem/develop' into feat/refacto…

53da8e4

…r-core-database

fix: missed a method rename here.

8453533

revert: changes to business repo tests asserting count.

27b9d75

fix: missing crucial line of code in 'DatabaseService.getForgedTransactionIds'

fix: failing tests in core-transaction-pool

bd5c1db

fix: failing tests in core-graphql

705b4ec

refactor: rename these files.

fdcdd87

test: re-enable this test until it's re-written.

32ef185

ghost added Status: Needs Review labels Feb 3, 2019

Merge remote-tracking branch 'arkecosystem/develop' into feat/refacto…

95b1c01

…r-core-database

ghost mentioned this pull request Feb 3, 2019

[Weekly Digest] Jan 28, 2019 - Feb 3, 2019 #2056

Closed

faustbrian added the Status: Blocked label Feb 4, 2019

Merge branch 'develop' into feat/refactor-core-database

543cfaf

faustbrian self-requested a review February 4, 2019 05:20

Merge branch 'develop' into feat/refactor-core-database

faca4e0

faustbrian reviewed Feb 4, 2019

View reviewed changes

paroxysm added 2 commits February 4, 2019 20:51

fix: startup issue in core-snapshots-cli(apparently it supports booti…

6a02a43

…ng up w/o a db connection) refactor: export a database-service factory for instantiating it. This prevents db flavor implementors from overriding the business repos which have expected implementations already.

Merge remote-tracking branch 'arkecosystem/develop' into feat/refacto…

0a091e1

…r-core-database

paroxysm added 2 commits February 4, 2019 20:59

fix: compilation failure here.

b7bc01e

Merge branch 'develop' into feat/refactor-core-database

2af2cbe

faustbrian changed the base branch from develop to 2.2 February 5, 2019 06:25

faustbrian merged commit 43db7c6 into ArkEcosystem:2.2 Feb 5, 2019

ghost removed Status: Blocked labels Feb 5, 2019

faustbrian added the Bounty: Tier 0 label Feb 5, 2019

faustbrian added a commit that referenced this pull request Feb 7, 2019

refactor(core-database): separate business-logic from data-layer logic (

4e41294

#2055)

vasild added a commit that referenced this pull request Feb 7, 2019

Merge remote-tracking branch 'ArkEcosystem/core/develop' into verify-…

2d5f335

…peer-state * ArkEcosystem/core/develop: refactor(core-database): separate business-logic from data-layer logic (#2055)

ghost mentioned this pull request Feb 11, 2019

[Weekly Digest] Feb 4, 2019 - Feb 10, 2019 #2078

Closed

paroxysm deleted the feat/refactor-core-database branch February 21, 2019 02:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Separate business-logic from data-layer logic in core-database/postgres#2055

Separate business-logic from data-layer logic in core-database/postgres#2055
faustbrian merged 22 commits intoArkEcosystem:2.2from
paroxysm:feat/refactor-core-database

paroxysm commented Feb 3, 2019 •

edited

Loading

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

faustbrian Feb 4, 2019

Uh oh!

paroxysm Feb 5, 2019

Uh oh!

paroxysm commented Feb 4, 2019 •

edited

Loading

Uh oh!

ghost commented Feb 5, 2019 •

edited by ghost

Loading

Uh oh!

ghost commented Feb 5, 2019 •

edited by faustbrian

Loading

Uh oh!

faustbrian commented Feb 5, 2019

Uh oh!

paroxysm commented Feb 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

paroxysm commented Feb 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Types of changes

Checklist

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

ghost commented Feb 3, 2019

Uh oh!

faustbrian Feb 4, 2019

Choose a reason for hiding this comment

Uh oh!

paroxysm Feb 5, 2019

Choose a reason for hiding this comment

Uh oh!

paroxysm commented Feb 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Feb 5, 2019 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Feb 5, 2019 • edited by faustbrian Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

faustbrian commented Feb 5, 2019

Uh oh!

paroxysm commented Feb 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

paroxysm commented Feb 3, 2019 •

edited

Loading

paroxysm commented Feb 4, 2019 •

edited

Loading

ghost commented Feb 5, 2019 •

edited by ghost

Loading

ghost commented Feb 5, 2019 •

edited by faustbrian

Loading