Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Conversation MLS verification status updating (WPB-3872) #2035

Merged

Conversation

borichellow
Copy link
Contributor

@borichellow borichellow commented Sep 6, 2023

What's new in this PR?

Implementing the machine for calculating the verification state of Conversation.

Issues

CoreCrypto can only "say" if MLS conversation is verified or not, while we need also to indicate when the conversation verification state was degraded (was verified but became not_verified).

Solutions

  • Store the conversation verification state, so we can compare to it the next state.
  • When something changes in conversation (adding/removing clients) check its state again and store a proper value.
  • Move the whole conversation verification logic into ConversationVerificationStatusHandlerand call it (start observing the state changes) every time ConversationDetails are observed (by observing that flow in ObserveConversationDetailsUseCase)

@borichellow borichellow added the WIP work in progress label Sep 6, 2023
@CLAassistant
Copy link

CLAassistant commented Sep 6, 2023

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023

Unit Test Results

   438 files   - 1     438 suites   - 1   2m 7s ⏱️ +7s
2 381 tests  - 7  2 277 ✔️  - 7  104 💤 ±0  0 ±0 

Results for commit d4d1ba7. ± Comparison against base commit 718a7e1.

♻️ This comment has been updated with latest results.

@datadog-wireapp
Copy link

datadog-wireapp bot commented Sep 6, 2023

Datadog Report

All test runs 9505750 🔗

2 Total Test Services: 1 Failed, 0 with New Flaky, 1 Passed

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Wall Time Branch View
kalium-ios 12 0 0 1932 67 9m 1.1s Link
kalium-jvm 0 0 0 1 0 1.89s Link

❌ Failed Tests (12)

This report shows up to 5 failed tests.

  • givenCurrentVerificationStatusIsDegraded_whenNewNotVerifiedStatusComes_thenDegradedStatusSaved[iosX64] - com.wire.kalium.logic.feature.conversation.ConversationVerificationStatusHandlerTest - Details

    Expand for error
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7af90]
     
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7af90]
     	at kotlin.Error#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
     	at kotlin.AssertionError#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:132)
     	at kotlinx.coroutines.test.UncompletedCoroutinesError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestScope.kt:325)
     	at kotlinx.coroutines.test.runTest$lambda$6$lambda$4$lambda$3#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:349)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.$<bridge-UNNN>invoke(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.InvokeOnCancelling.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:1431)
     ...
    
  • givenCurrentVerificationStatusIsVerified_whenNewNotVerifiedStatusComes_thenDegradedStatusSaved[iosX64] - com.wire.kalium.logic.feature.conversation.ConversationVerificationStatusHandlerTest - Details

    Expand for error
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7ed50]
     
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7ed50]
     	at kotlin.Error#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
     	at kotlin.AssertionError#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:132)
     	at kotlinx.coroutines.test.UncompletedCoroutinesError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestScope.kt:325)
     	at kotlinx.coroutines.test.runTest$lambda$6$lambda$4$lambda$3#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:349)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.$<bridge-UNNN>invoke(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.InvokeOnCancelling.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:1431)
     ...
    
  • givenCurrentVerificationStatusIsVerified_whenNewVerifiedStatusComes_thenNothingUpdated[iosX64] - com.wire.kalium.logic.feature.conversation.ConversationVerificationStatusHandlerTest - Details

    Expand for error
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a72410]
     
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a72410]
     	at kotlin.Error#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
     	at kotlin.AssertionError#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:132)
     	at kotlinx.coroutines.test.UncompletedCoroutinesError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestScope.kt:325)
     	at kotlinx.coroutines.test.runTest$lambda$6$lambda$4$lambda$3#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:349)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.$<bridge-UNNN>invoke(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.InvokeOnCancelling.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:1431)
     ...
    
  • givenFailureWhileGettingUserInformedAboutDegradedStatus_whenInvokingWithDegradedStatus_thenInformedFlagNotSetFalse[iosX64] - com.wire.kalium.logic.feature.conversation.ConversationVerificationStatusHandlerTest - Details

    Expand for error
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7da50]
     
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7da50]
     	at kotlin.Error#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
     	at kotlin.AssertionError#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:132)
     	at kotlinx.coroutines.test.UncompletedCoroutinesError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestScope.kt:325)
     	at kotlinx.coroutines.test.runTest$lambda$6$lambda$4$lambda$3#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:349)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.$<bridge-UNNN>invoke(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.InvokeOnCancelling.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:1431)
     ...
    
  • givenMLSConversationAndInformedAboutDegraded_whenInvokingWithVerifiedStatus_thenInformedFlagSetTrue[iosX64] - com.wire.kalium.logic.feature.conversation.ConversationVerificationStatusHandlerTest - Details

    Expand for error
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7d410]
     
     kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing, there were active child jobs: [DispatchedCoroutine{Completing}@a7a7d410]
     	at kotlin.Error#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:14)
     	at kotlin.AssertionError#<init>(/opt/buildAgent/work/f43969c6214a19e7/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Exceptions.kt:132)
     	at kotlinx.coroutines.test.UncompletedCoroutinesError#<init>(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestScope.kt:325)
     	at kotlinx.coroutines.test.runTest$lambda$6$lambda$4$lambda$3#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:349)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.test.$runTest$lambda$6$lambda$4$lambda$3$FUNCTION_REFERENCE$10.$<bridge-UNNN>invoke(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-test/common/src/TestBuilders.kt:333)
     	at kotlinx.coroutines.InvokeOnCancelling.invoke#internal(/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:1431)
     ...
    

@typfel
Copy link
Member

typfel commented Sep 8, 2023

@borichellow do you want feedback on this? If so add a description of what you've done.

@codecov-commenter
Copy link

codecov-commenter commented Sep 22, 2023

Codecov Report

Merging #2035 (d4d1ba7) into develop (718a7e1) will decrease coverage by 0.06%.
The diff coverage is 75.00%.

@@              Coverage Diff              @@
##             develop    #2035      +/-   ##
=============================================
- Coverage      57.94%   57.88%   -0.06%     
  Complexity        24       24              
=============================================
  Files           1016     1014       -2     
  Lines          38173    38215      +42     
  Branches        3472     3469       -3     
=============================================
+ Hits           22119    22121       +2     
- Misses         14562    14604      +42     
+ Partials        1492     1490       -2     
Files Coverage Δ
...ire/kalium/logic/data/conversation/Conversation.kt 92.94% <100.00%> (+0.13%) ⬆️
...gic/data/conversation/MLSConversationRepository.kt 85.56% <100.00%> (ø)
...otlin/com/wire/kalium/logic/data/web/WebMappers.kt 36.80% <100.00%> (+0.50%) ⬆️
...ion/MLSConversationsVerificationStatusesHandler.kt 100.00% <100.00%> (ø)
...um/persistence/dao/conversation/ConversationDAO.kt 100.00% <ø> (ø)
...persistence/dao/conversation/ConversationEntity.kt 98.03% <100.00%> (+0.08%) ⬆️
...istence/dao/conversation/ConversationViewEntity.kt 97.61% <100.00%> (+0.05%) ⬆️
...tlin/com/wire/kalium/persistence/db/TableMapper.kt 100.00% <100.00%> (ø)
...lium/logic/data/connection/ConnectionRepository.kt 65.94% <0.00%> (-0.49%) ⬇️
...lium/logic/data/conversation/ConversationMapper.kt 55.32% <75.00%> (+0.64%) ⬆️
... and 3 more

... and 10 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 718a7e1...d4d1ba7. Read the comment docs.

@typfel
Copy link
Member

typfel commented Sep 22, 2023

So checking that've understood the implementation correctly

We have the existing ObserveConversationDetailsUseCase which is the entry point which starts the ConversationVerificationStatusHandler.

ConversationVerificationStatusHandler does:

observes the epoch flow ->
ask CC if verified or not ->
fetch previous verification status ->
derive new verification status ->
insert system message if necessary

Now some questions:

  1. Do we only want to update the verification status when conversation if it's displayed on screen? Shouldn't
    it be driven from kalium itself and not by the UI?

  2. Wouldn't we potentially lose state transistion for conversations
    which are not observed.

For example, given clients are added add and then removed again:

verified
not verified
verified

Wouldn't we fail to report that the conversation was unverified for time period?

@borichellow
Copy link
Contributor Author

So checking that've understood the implementation correctly

We have the existing ObserveConversationDetailsUseCase which is the entry point which starts the ConversationVerificationStatusHandler.

ConversationVerificationStatusHandler does:

observes the epoch flow -> ask CC if verified or not -> fetch previous verification status -> derive new verification status -> insert system message if necessary

Yes, you understood the flow perfectly.

  1. Do we only want to update the verification status when conversation if it's displayed on screen? Shouldn't
    it be driven from kalium itself and not by the UI?

Well, observing these statuses for ALL the users conversations may have tragical app performance decreasing.
The right solution here is corresponding web-socket event, but we don't have it, so I'm just doing my best with what I have :)

For example, given clients are added add and then removed again:

verified
not verified
verified

Wouldn't we fail to report that the conversation was unverified for time period?

True, user will lose "not verified" status in that scenario, but I believe it's not so important, as user was not texting anything to that conversation while those changes happened, so verification status is not so important for him for that period of time.

The main goal is to inform user when the conversation he's going to text/call is not secure anymore.

@@ -39,7 +39,7 @@ class ProtocolInfoMapperImpl(
Conversation.ProtocolInfo.MLS.GroupState.valueOf(protocolInfo.groupState.name),
protocolInfo.epoch,
protocolInfo.keyingMaterialLastUpdate,
Conversation.CipherSuite.fromTag(protocolInfo.cipherSuite.cipherSuiteTag)
Conversation.CipherSuite.fromTag(protocolInfo.cipherSuite.cipherSuiteTag),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Conversation.CipherSuite.fromTag(protocolInfo.cipherSuite.cipherSuiteTag),
Conversation.CipherSuite.fromTag(protocolInfo.cipherSuite.cipherSuiteTag)

.distinctUntilChanged()
.onlyRight()
.mapLatest { newStatus ->
val currentStatus = conversationRepository.getConversationVerificationStatus(conversationId)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: do we need to call CC anymore? since we update/fetch the state based on the epoch changes? here simply can't we only observe the value in the db?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I got your question.
We observe epoch changes, but it only triggers us that something was changed and we need to check if the Verification Status changed (by calling CC).

@@ -76,7 +78,7 @@ data class ConversationEntity(
enum class MutedStatus { ALL_ALLOWED, ONLY_MENTIONS_AND_REPLIES_ALLOWED, MENTIONS_MUTED, ALL_MUTED }

sealed class ProtocolInfo {
object Proteus : ProtocolInfo()
data object Proteus : ProtocolInfo()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
data object Proteus : ProtocolInfo()
object Proteus : ProtocolInfo()

Copy link
Member

@mchenani mchenani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added one question and some other tiny comments.

@typfel
Copy link
Member

typfel commented Sep 25, 2023

So checking that've understood the implementation correctly
We have the existing ObserveConversationDetailsUseCase which is the entry point which starts the ConversationVerificationStatusHandler.
ConversationVerificationStatusHandler does:
observes the epoch flow -> ask CC if verified or not -> fetch previous verification status -> derive new verification status -> insert system message if necessary

Yes, you understood the flow perfectly.

  1. Do we only want to update the verification status when conversation if it's displayed on screen? Shouldn't
    it be driven from kalium itself and not by the UI?

Well, observing these statuses for ALL the users conversations may have tragical app performance decreasing. The right solution here is corresponding web-socket event, but we don't have it, so I'm just doing my best with what I have :)

But we will never a web-socket event since this all driven by the clients without any knowledge from the server, no?

Why do think this would have such a big performance impact? Commits are not arriving constantly in conversations. We don't need a a separate observer for each conversation we can one observe for all conversations.

For example, given clients are added add and then removed again:
verified
not verified
verified
Wouldn't we fail to report that the conversation was unverified for time period?

True, user will lose "not verified" status in that scenario, but I believe it's not so important, as user was not texting anything to that conversation while those changes happened, so verification status is not so important for him for that period of time.

The main goal is to inform user when the conversation he's going to text/call is not secure anymore.

Ok if that's the only thing we are concerned about. But for proteus verification we also tried to insert the system message at the "right" point time so that the user can see which messages were sent or received when the conversation was in verified/degraded/unverified state.

@borichellow
Copy link
Contributor Author

@typfel Updated the PR, no it run observation in SessionsScope once for each user, looks much better i believe :)

Copy link
Member

@mchenani mchenani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

Comment on lines 61 to 72
if ((newStatus == VerificationStatus.NOT_VERIFIED && currentStatus == VerificationStatus.DEGRADED) ||
newStatus == currentStatus
) {
return@collect
}

if (newStatus == VerificationStatus.NOT_VERIFIED && currentStatus == VerificationStatus.VERIFIED) {
conversationRepository.updateVerificationStatus(VerificationStatus.DEGRADED, conversation.conversation.id)
notifyUserAboutStateChanges(conversation.conversation.id, VerificationStatus.DEGRADED)
} else {
conversationRepository.updateVerificationStatus(newStatus, conversation.conversation.id)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: extract into private function

@borichellow borichellow added this pull request to the merge queue Sep 28, 2023
Merged via the queue into develop with commit 2497ff6 Sep 28, 2023
16 of 17 checks passed
@borichellow borichellow deleted the feat/conversation_mls_verification_status_updating branch September 28, 2023 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants