Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better stream eventing #537

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

Better stream eventing #537

wants to merge 6 commits into from

Conversation

mqus
Copy link
Collaborator

@mqus mqus commented Apr 18, 2021

This changes the datatype of the events for streaming to Set<String> and implements two features that benefit from that:

  • If a delete/update also changes other tables via foreign-key relationships, then streams on those entities also get updated, depending on the columns and types of relationships, even transitively.
  • Changes made in a transaction will trigger all affected tables at the end (and not before), and only if the transaction succeeded.

What did not change: the handling of streams of views. They will still get updated by any changes in the database, since those dependencies can only be resolved with a query parser/analyzer.

To summarize, this will cause better stream updates for entities with FK-relationships and transaction events, and while also outputting less events than before.

It is probably best to review this commit-by-commit.

Addresses #373 partially.
Closes #360

@codecov
Copy link

codecov bot commented Apr 18, 2021

Codecov Report

Merging #537 (a47443c) into develop (e473c4b) will increase coverage by 1.21%.
The diff coverage is 100.00%.

❗ Current head a47443c differs from pull request most recent head b394443. Consider uploading reports for the commit b394443 to get more accurate results
Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #537      +/-   ##
===========================================
+ Coverage    90.07%   91.28%   +1.21%     
===========================================
  Files           74       10      -64     
  Lines         1853      195    -1658     
===========================================
- Hits          1669      178    -1491     
+ Misses         184       17     -167     
Flag Coverage Δ
floor 91.28% <100.00%> (+0.85%) ⬆️
floor_generator ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
floor/lib/src/adapter/deletion_adapter.dart 100.00% <100.00%> (ø)
floor/lib/src/adapter/insertion_adapter.dart 100.00% <100.00%> (ø)
floor/lib/src/adapter/query_adapter.dart 97.61% <100.00%> (+2.61%) ⬆️
floor/lib/src/adapter/update_adapter.dart 100.00% <100.00%> (ø)
floor_generator/lib/value_object/queryable.dart
...ator/lib/misc/extension/dart_object_extension.dart
...or_generator/lib/processor/database_processor.dart
floor_generator/lib/misc/type_utils.dart
...ator/lib/processor/error/view_processor_error.dart
...nerator/lib/processor/update_method_processor.dart
... and 53 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e473c4b...b394443. Read the comment docs.

@mqus mqus added feature Request and implementation of a feature (release drafter) refactoring Improvement without changing functionality labels Apr 18, 2021
@mqus
Copy link
Collaborator Author

mqus commented Apr 23, 2021

I just noticed that I missed @inserts onConflict: replace and similar options, which also cause cascading changes, I'll have to consider this.

Copy link
Collaborator

@vitusortner vitusortner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll continue the review ASAP. So far, I left some questions.

]) : assert(entityName.isNotEmpty),
assert(primaryKeyColumnName.isNotEmpty),
_database = database,
_entityName = entityName,
_primaryKeyColumnNames = primaryKeyColumnName,
_valueMapper = valueMapper,
_changeListener = changeListener;
_changeHandler = changeHandler ?? (() {/* do nothing */});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ No changes necessarily needed. Only asking out of curiosity: We can avoid this default parameter by making _changeHandler nullable and calling it this way _changeHandler?.call(). What made you rely on default parameters (empty function body) instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking for null on each invocation might incur some very minor runtime cost but apart from that I have no real reason. If you prefer this version then I'll change it.

@@ -42,7 +42,7 @@ class InsertionAdapter<T> {
);
}
await batch.commit(noResult: true);
_changeListener?.add(_entityName);
_changeListener?.add({_entityName});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ We now have these two approaches in our codebase where we feed into the change listener directly in the adapter (insertion_adapter) and where we feed into the change listener indirectly with the help of lambdas (update_adapter, deletion_adapter). Does it make sense to align all of them?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the insertion_adapter also needs lambdas because of the onconflict:update issue... I thought I committed that already but it seems like I didn't...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah right, I didn't do this because the onConflict clause can be different for each call of insertX, altering the entities which have to be notified... And I could not decide what is best.

There are basically three main options here:

  1. always assume onConflict:Update if it occurs even once for a InsertionAdapter.insert call
  2. give the InsertionAdapter two different sets/lambdas via the constructor and it will choose which one to use for triggering updates, based on the onConflict clause given in the InsertionAdapter.insert call
  3. insert the lambda/set via the invocation of the InsertionAdapter.insert call and not in the constructor

I'm not sure which is best here, but 1 seems to be suboptimal to me.

///
/// Traverses the given dependency map (Parent:[(ForeignKey:Child)]) and
/// tries to figure out which entities an update of [e] could at most change.
///It excludes all children with a [ForeignKeyAction.noAction] and [ForeignKeyAction.restrict]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
///It excludes all children with a [ForeignKeyAction.noAction] and [ForeignKeyAction.restrict]
/// It excludes all children with a [ForeignKeyAction.noAction] and [ForeignKeyAction.restrict]

class ForeignKeyMap {
final Map<String, Map<ForeignKey, Entity>> fkDependencies;

ForeignKeyMap.fromEntities(Iterable<Entity> stuff)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ForeignKeyMap.fromEntities(Iterable<Entity> stuff)
ForeignKeyMap.fromEntities(Iterable<Entity> entities)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops :D

@saharvx9
Copy link

following working awsome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Request and implementation of a feature (release drafter) refactoring Improvement without changing functionality
Development

Successfully merging this pull request may close these issues.

handle changeListener inputs in transactions correctly
3 participants