Fix data stream batching #32
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This is sufficient for a quick workaround (given that it works), but the real underlying issue with the entire data streams implementation is that it was blindly copied over from core and wasn't given much thought. The real question you have to ask here is: what does In CAWS, we don't need this representation, as we can just use the batch as is, and dump the sequences in the db (given that its conditions are met). |
davidscavnicky
commented
Apr 29, 2024
xelahalo
requested changes
Apr 29, 2024
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
davidscavnicky
commented
Apr 29, 2024
src/test/kotlin/dk/cachet/carp/webservices/data/service/CoreDataStreamServiceTest.kt
Outdated
Show resolved
Hide resolved
davidscavnicky
commented
Apr 29, 2024
davidscavnicky
commented
Apr 29, 2024
xelahalo
force-pushed
the
feature/data-mismatch
branch
from
c5f3e10
to
372d9c8
Compare
xelahalo
approved these changes
May 2, 2024
xelahalo
changed the title
xelahalo
added a commit
that referenced
this pull request
May 7, 2024
* Check if claims exist before account parsing * Apply runBlocking on repository level We apply runBlocking again, as it turns out, that our JDBC connector cannot handle suspend functions,and is always doing blocking calls. We will need to change the persistence layer if we want to deal with this properly. * Add remove option to deployment script * Add participant as a default role in keycloak * Keycloakify: theme info page and edit Privacy Policy text (#14) * Fix some text * Make info page use Carp theme * Upgrade to kotlin 1.9.23 and related dependencies * Upgrade CARP Core to 1.2.0 * Adjust default token lifespans * Clean up environment urls * Remove caffeine caching * Add controller level UUID conversion * Prepare account services for anonymous participants * Generate anonymous participants * Fix docker compose CAWS port exposing * Toggle off email as username * Give unique name to anonymous accounts csv * Redirect to parameter instead of portal for anonymous accounts As a workaround we make magic-links redirect to the study instead of the portal, as the portal is not in an adequate state to be able to parse and do the redirects. This will have to be reverted (and given more thought) after the ICAT study is done and there is a need for anonymous deployments for generic web/mobile-hosted studies. * Disable statistics endpoint * Rewrite authorization to use OAuth claims * Authorize ApplicationServices * Readd logging * Remove researcher reference from study * Bump versions - gradle from 7.4.2 to 8.7 - spring boot from 3.1.0 to 3.2.4 - spring addons from 6.1.2 to 7.6.12 - hibernate from 6.2.7 to 6.4.4 * Extend keycloak config to include claims for non-core endpoints * Use IO dispatcher for repository calls * Upgrade Keycloak theme (#42) * Update keycloakify to v9 * Migrate existing pages * Remove duplicate protocol mapper * Update theme name * Built new jar file * Fix Dockerfile gradle version * Add missing client scope mapper to default realm * Fix typo in application properties * Keycloak theme: fix broken button text on info page (#43) * Change required claims for consent documents (#45) * Make security expressions accept nullable values * Make every claim String-based The default Spring Addons authority converters can't deal with non-string values. * Stop granting ManageDeployment Claims to Researchers * Get authentication from the original thread when changing claims from non-suspend functions * Make study owners be able to get all consent documents again * Improve button style for Input page and overall responsiveness (#44) * Improve button styles for info page * Build updated .jar file for KC theme * Revoke claims through authorizers when resource gets deleted (#46) This solution will not revoke the claims if the resources get deleted through a call from the services event bus. However that shouldn't be a big issue, if one doesn't use the claims in a way that presumes the existence of the referenced objects. * Parallelize anonymous account request (#49) * Add parallelization for anonymous accounts * Optimize study overview * Add getting protocols overview (#22) Co-authored-by: xelahalo <olahalex8@gmail.com> * Add created by to study overview (#50) * Fix data stream batching (#32) Co-authored-by: xelahalo <olahalex8@gmail.com> * Use MS Teams for notifications (#54) * add basic notification for MS-teams * add notification channels corrsponding to MS teams channel, renamed/removed notificattions to Slack channels * add teams channel notification for development server/profile * Update .env * rename methods * update notification related comments * Release 1.2.0 --------- Co-authored-by: Aamir Farooq <SlimShadyIAm@users.noreply.github.com> Co-authored-by: David Scavnicky <109144925+davidscavnicky@users.noreply.github.com> Co-authored-by: Yuan Chen <10835085+yuanchen233@users.noreply.github.com>
davidscavnicky
added a commit
that referenced
this pull request
May 31, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
data was gathered in cyclic manner MutableDataStreamBatch, inserted class into appendToDataStreams() so upon next appendToDataStream the class is initialized again.
The issue and discussion:
#16