Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native auth safe logging test #2092

Merged
merged 39 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7b69ea5
Safe logging test
Yuki-YuXin Apr 29, 2024
93d0805
minor fix
Yuki-YuXin Apr 29, 2024
66679a6
Use regex rules to filter the logs
Yuki-YuXin Apr 29, 2024
3e545b2
Modify rules and pass tests
Yuki-YuXin Apr 29, 2024
aad4fc5
Update tag to use regex
Yuki-YuXin Apr 29, 2024
c8072e4
Modify regex rules and use any() in tag log.
Yuki-YuXin Apr 30, 2024
d3df120
spy(ILoggerCallback) + assertFalse(regex.matches)
Yuki-YuXin Apr 30, 2024
21d4910
variable order
Yuki-YuXin Apr 30, 2024
1fead78
assertFalse(spyLoggerCallback.failCalled) in every test to fail test.
Yuki-YuXin Apr 30, 2024
0f67b09
remove blank line
Yuki-YuXin Apr 30, 2024
761a77f
Java version sample
Yuki-YuXin May 1, 2024
c6a08a6
Merge branch 'dev' into yuki/logging-check
Yuki-YuXin May 1, 2024
1d542e6
Revert "Java version sample"
Yuki-YuXin May 1, 2024
9939418
Fix for password regex rule
Yuki-YuXin May 1, 2024
248a338
Narrow down the regex scopes and pass tests.
Yuki-YuXin May 1, 2024
4c2ba7c
First version for CountDownLatch but the callback takes too much time…
Yuki-YuXin May 2, 2024
07498f3
Fix error which caused by wrong order.
Yuki-YuXin May 2, 2024
07bd03b
Use Parameterized for allowPII=true/false but has issue with latch
Yuki-YuXin May 3, 2024
e91bb0d
Fix the latch bug issue
Yuki-YuXin May 3, 2024
dd407cd
Fix bug on log callback part to check both message and containsPII item
Yuki-YuXin May 3, 2024
71be953
Clean the code logic in the callback part
Yuki-YuXin May 3, 2024
3f8e788
Remove withContext block over latch.await
Yuki-YuXin May 3, 2024
0c513ff
assertTrue(containsPII) => fail()
Yuki-YuXin May 3, 2024
a7d6908
Merge branch 'dev' into yuki/logging-check
Yuki-YuXin May 7, 2024
7404a79
Use verify instead of fail
Yuki-YuXin May 8, 2024
eedb6b1
Add containPii check
Yuki-YuXin May 8, 2024
b6a5c76
Rename variables
Yuki-YuXin May 8, 2024
c061727
Update Java version
Yuki-YuXin May 8, 2024
1a7d1bf
Minor in Kotlin version: fix any()->anyBoolean()
Yuki-YuXin May 8, 2024
c539cfd
Create a component for the logger safe checking
Yuki-YuXin May 10, 2024
7750639
JWT token regex rule
Yuki-YuXin May 10, 2024
6816d2d
Remove redundant blank line
Yuki-YuXin May 10, 2024
0a05fff
Remove redundant remove logger in Java version
Yuki-YuXin May 10, 2024
3c2dc77
Add black line for the new class
Yuki-YuXin May 10, 2024
f12c022
Use the updated secret
Yuki-YuXin May 13, 2024
7c97e54
Use the updated secret
Yuki-YuXin May 13, 2024
9bcd81c
Revert "Use the updated secret"
Yuki-YuXin May 13, 2024
bd552ed
Revert "Use the updated secret"
Yuki-YuXin May 13, 2024
e03b9da
Merge branch 'dev' into yuki/logging-check
Yuki-YuXin May 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion azure-pipelines/pull-request-validation/pr-msal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
clean: true
submodules: recursive
persistCredentials: True
- template: azure-pipelines/templates/steps/automation-cert.yml@common
- task: JavaToolInstaller@0
displayName: Use Java 11
inputs:
Expand All @@ -66,7 +67,7 @@ jobs:
- task: Gradle@2
displayName: Run Unit tests
inputs:
tasks: msal:testLocalDebugUnitTest -Plabtest -PlabSecret=$(LabVaultAppSecret) -ProbolectricSdkVersion=${{variables.robolectricSdkVersion}}
tasks: msal:testLocalDebugUnitTest -Plabtest -PlabSecret=$(LabVaultAppCert) -ProbolectricSdkVersion=${{variables.robolectricSdkVersion}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is opening PR (and has not merged) from MSAL to update the expired secret. I include their changes to see if that's the reason for the failed pipeline verification. I personally think we need to wait that PR to be merged in order to merge our work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. Let's not introduce changes with this PR that don't belong here. Let's mark this PR as "draft" in the meantime.

javaHomeSelection: $(BuildParameters.javaHomeSelection)
jdkVersion: 1.11
- job: spotbugs
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,11 @@ public void log(String tag, com.microsoft.identity.common.internal.logging.Logge
mExternalLogger = externalLogger;
}

public synchronized void removeExternalLogger() {
mExternalLogger = null;
}


/**
* Enable/Disable the Android logcat logging. By default, the sdk enables it.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import android.app.Activity;
import android.content.Context;

import com.microsoft.identity.client.ILoggerCallback;
import com.microsoft.identity.client.Logger;
import com.microsoft.identity.client.PublicClientApplication;
import com.microsoft.identity.client.e2e.shadows.ShadowAndroidSdkStorageEncryptionManager;
Expand Down Expand Up @@ -74,6 +75,7 @@
import com.microsoft.identity.common.java.util.ResultFuture;
import com.microsoft.identity.internal.testutils.TestUtils;
import com.microsoft.identity.nativeauth.statemachine.states.SignUpPasswordRequiredState;
import com.microsoft.identity.nativeauth.utils.LoggerCheckHelper;

import org.junit.After;
import org.junit.AfterClass;
Expand All @@ -82,13 +84,17 @@
import org.junit.Ignore;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.robolectric.RobolectricTestRunner;
import org.mockito.MockitoAnnotations;
import org.robolectric.ParameterizedRobolectricTestRunner;
import org.robolectric.annotation.Config;
import org.robolectric.annotation.LooperMode;

import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.util.Arrays;
import java.util.Collection;
import java.util.UUID;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
Expand All @@ -106,7 +112,7 @@
import static org.mockito.Mockito.spy;
import static org.robolectric.annotation.LooperMode.Mode.LEGACY;

@RunWith(RobolectricTestRunner.class)
@RunWith(ParameterizedRobolectricTestRunner.class)
@LooperMode(LEGACY)
@Config(shadows = {ShadowAndroidSdkStorageEncryptionManager.class})
public class NativeAuthPublicClientApplicationJavaTest extends PublicClientApplicationAbstractTest {
Expand All @@ -115,11 +121,20 @@ public class NativeAuthPublicClientApplicationJavaTest extends PublicClientAppli
private IPlatformComponents components;
private Activity activity;
private INativeAuthPublicClientApplication application;
private LoggerCheckHelper loggerCheckHelper;
private final String username = "user@email.com";
private final String invalidUsername = "invalidUsername";
private final char[] password = "verySafePassword".toCharArray();
private final String code = "1234";
private final String emptyString = "";
private final boolean allowPII;

public NativeAuthPublicClientApplicationJavaTest(boolean allowPII) {
this.allowPII = allowPII;
}

@Mock
private ILoggerCallback externalLogger;


@Override
Expand All @@ -137,21 +152,26 @@ public static void tearDownClass() {
setUseMockApiForNativeAuth(false);
}

@ParameterizedRobolectricTestRunner.Parameters
public static Collection<Boolean> data() {
return Arrays.asList(true, false);
}

@Before
public void setup() {
MockitoAnnotations.initMocks(this);
context = ApplicationProvider.getApplicationContext();
components = AndroidPlatformComponentsFactory.createFromContext(context);
activity = Mockito.mock(Activity.class);
loggerCheckHelper = new LoggerCheckHelper(externalLogger, allowPII);
Mockito.when(activity.getApplicationContext()).thenReturn(context);
setupPCA();
Logger.getInstance().setEnableLogcatLog(true);
Logger.getInstance().setEnablePII(true);
Logger.getInstance().setLogLevel(Logger.LogLevel.VERBOSE);
CommandDispatcherHelper.clear();
}

@After
public void cleanup() {
loggerCheckHelper.checkSafeLogging();
AcquireTokenTestHelper.setAccount(null);
// remove everything from cache after test ends
TestUtils.clearCache(SHARED_PREFERENCES_NAME);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,29 @@ package com.microsoft.identity.nativeauth
import android.app.Activity
import android.content.Context
import androidx.test.core.app.ApplicationProvider
import com.microsoft.identity.client.ILoggerCallback
import com.microsoft.identity.client.Logger
import com.microsoft.identity.client.PublicClientApplication
import com.microsoft.identity.client.e2e.shadows.ShadowAndroidSdkStorageEncryptionManager
import com.microsoft.identity.client.e2e.tests.PublicClientApplicationAbstractTest
import com.microsoft.identity.client.e2e.utils.AcquireTokenTestHelper
import com.microsoft.identity.client.exception.MsalClientException
import com.microsoft.identity.client.exception.MsalException
import com.microsoft.identity.common.components.AndroidPlatformComponentsFactory
import com.microsoft.identity.common.internal.controllers.CommandDispatcherHelper
import com.microsoft.identity.common.java.exception.BaseException
import com.microsoft.identity.common.java.interfaces.IPlatformComponents
import com.microsoft.identity.common.java.nativeauth.BuildValues
import com.microsoft.identity.common.java.util.ResultFuture
import com.microsoft.identity.common.nativeauth.MockApiEndpoint
import com.microsoft.identity.common.nativeauth.MockApiResponseType
import com.microsoft.identity.common.nativeauth.MockApiUtils.Companion.configureMockApi
import com.microsoft.identity.internal.testutils.TestUtils
import com.microsoft.identity.nativeauth.statemachine.errors.ErrorTypes
import com.microsoft.identity.nativeauth.statemachine.errors.GetAccessTokenError
import com.microsoft.identity.nativeauth.statemachine.errors.ResetPasswordError
import com.microsoft.identity.nativeauth.statemachine.errors.ResetPasswordSubmitPasswordError
import com.microsoft.identity.nativeauth.statemachine.errors.SignInContinuationError
import com.microsoft.identity.nativeauth.statemachine.errors.SignInError
import com.microsoft.identity.nativeauth.statemachine.errors.SignUpError
import com.microsoft.identity.nativeauth.statemachine.errors.SignUpSubmitAttributesError
Expand All @@ -48,20 +62,9 @@ import com.microsoft.identity.nativeauth.statemachine.results.SignInResult
import com.microsoft.identity.nativeauth.statemachine.results.SignOutResult
import com.microsoft.identity.nativeauth.statemachine.results.SignUpResendCodeResult
import com.microsoft.identity.nativeauth.statemachine.results.SignUpResult
import com.microsoft.identity.common.components.AndroidPlatformComponentsFactory
import com.microsoft.identity.common.internal.controllers.CommandDispatcherHelper
import com.microsoft.identity.common.nativeauth.MockApiEndpoint
import com.microsoft.identity.common.nativeauth.MockApiResponseType
import com.microsoft.identity.common.nativeauth.MockApiUtils.Companion.configureMockApi
import com.microsoft.identity.common.java.exception.BaseException
import com.microsoft.identity.common.java.interfaces.IPlatformComponents
import com.microsoft.identity.common.java.nativeauth.BuildValues
import com.microsoft.identity.common.java.util.ResultFuture
import com.microsoft.identity.internal.testutils.TestUtils
import com.microsoft.identity.nativeauth.statemachine.errors.ErrorTypes
import com.microsoft.identity.nativeauth.statemachine.states.SignInContinuationState
import com.microsoft.identity.nativeauth.utils.LoggerCheckHelper
import com.microsoft.identity.nativeauth.utils.mockCorrelationId
import com.microsoft.identity.nativeauth.statemachine.errors.SignInContinuationError
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.test.runTest
Expand All @@ -76,35 +79,55 @@ import org.junit.BeforeClass
import org.junit.Ignore
import org.junit.Test
import org.junit.runner.RunWith
import org.mockito.ArgumentMatcher
import org.mockito.ArgumentMatchers.anyBoolean
import org.mockito.Mock
import org.mockito.Mockito
import org.mockito.MockitoAnnotations
import org.mockito.kotlin.any
import org.mockito.kotlin.argThat
import org.mockito.kotlin.eq
import org.mockito.kotlin.mock
import org.mockito.kotlin.never
import org.mockito.kotlin.spy
import org.mockito.kotlin.verify
import org.mockito.kotlin.whenever
import org.robolectric.RobolectricTestRunner
import org.robolectric.ParameterizedRobolectricTestRunner
import org.robolectric.annotation.Config
import java.io.File
import java.util.UUID
import java.util.concurrent.ExecutionException
import java.util.concurrent.TimeUnit
import java.util.concurrent.TimeoutException


@ExperimentalCoroutinesApi
@RunWith(RobolectricTestRunner::class)
@RunWith(ParameterizedRobolectricTestRunner::class)
@Config(shadows = [ShadowAndroidSdkStorageEncryptionManager::class])
class NativeAuthPublicClientApplicationKotlinTest : PublicClientApplicationAbstractTest() {
class NativeAuthPublicClientApplicationKotlinTest(private val allowPII: Boolean) : PublicClientApplicationAbstractTest() {
private lateinit var context: Context
private lateinit var components: IPlatformComponents
private lateinit var activity: Activity
private lateinit var application: INativeAuthPublicClientApplication
private lateinit var loggerCheckHelper: LoggerCheckHelper
private val username = "user@email.com"
private val invalidUsername = "invalidUsername"
private val password = "verySafePassword".toCharArray()
private val code = "1234"
private val emptyString = ""

@Mock
private lateinit var externalLogger: ILoggerCallback

override fun getConfigFilePath() = "src/test/res/raw/native_auth_native_only_test_config.json"

companion object {
@JvmStatic
@ParameterizedRobolectricTestRunner.Parameters
fun data(): Collection<Boolean> {
return listOf(true, false)
}

@BeforeClass
@JvmStatic
fun setupClass() {
Expand All @@ -120,16 +143,19 @@ class NativeAuthPublicClientApplicationKotlinTest : PublicClientApplicationAbstr

@Before
override fun setup() {
MockitoAnnotations.initMocks(this)
context = ApplicationProvider.getApplicationContext()
components = AndroidPlatformComponentsFactory.createFromContext(context)
activity = Mockito.mock(Activity::class.java)
loggerCheckHelper = LoggerCheckHelper(externalLogger, allowPII)
whenever(activity.applicationContext).thenReturn(context)
setupPCA()
CommandDispatcherHelper.clear()
}

@After
fun cleanup() {
loggerCheckHelper.checkSafeLogging()
AcquireTokenTestHelper.setAccount(null)
// remove everything from cache after test ends
TestUtils.clearCache(SHARED_PREFERENCES_NAME)
Expand Down Expand Up @@ -424,6 +450,7 @@ class NativeAuthPublicClientApplicationKotlinTest : PublicClientApplicationAbstr
)
val result = continuationTokenState.signIn(scopes = null)
assertTrue(result is SignInContinuationError)

}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package com.microsoft.identity.nativeauth.utils

import com.microsoft.identity.client.ILoggerCallback
import com.microsoft.identity.client.Logger
import org.mockito.ArgumentMatcher
import org.mockito.ArgumentMatchers
import org.mockito.kotlin.any
import org.mockito.kotlin.argThat
import org.mockito.kotlin.eq
import org.mockito.kotlin.never
import org.mockito.kotlin.verify

class LoggerCheckHelper(private val externalLogger: ILoggerCallback, private val allowPII: Boolean) {

private val sensitivePIIMessages = listOf(
"""(?<![\[\(])["]password["][:=]?(?![\]\)\}])""", // '"password":' '"password"=' exclude 'password' '"challengeType":["password"]' '"challenge_type":"password"}'
"""(?<![\s\?\(])(code)[:=]""", // 'code:' 'code=' exclude 'codeLength' 'error?code'
"""(?<![\(])continuationToken[:=]""",
"""(?<![\(])attributes[:=]""",
"""(?i)\b(accessToken|access_token)[:=]""", // access_token, accessToken
"""(?i)\b(refreshToken|refresh_token)[:=]""",
"""(?i)\b(idToken|id_token)[:=]""",
"""(?i)\b(continuation_token)[:=]""",
"""^[A-Za-z0-9_-]+\\.[A-Za-z0-9_-]+\\.[A-Za-z0-9_-]+$""" // JWT token
)
private val permittedPIIMessages = listOf(
"""(?<![\(])username[:=]""",
"""(?i)\b(challengeTargetLabel|challenge_target_label)[:=]"""
)

init {
setupLogger()
}

private fun setupLogger() {
Logger.getInstance().setLogLevel(Logger.LogLevel.INFO)
Logger.getInstance().setEnablePII(allowPII)
Logger.getInstance().setExternalLogger(externalLogger)
}

private fun clearLogger() {
Logger.getInstance().removeExternalLogger()
}

fun checkSafeLogging() {
var allowList = listOf<String>()
val disableList: List<String>

if (allowPII) {
allowList = permittedPIIMessages
disableList = sensitivePIIMessages
} else {
disableList = sensitivePIIMessages + permittedPIIMessages
}

allowList.forEach { regex ->
verifyLogCouldContain(regex)
}
disableList.forEach { regex ->
verifyLogDoesNotContain(regex)
}

clearLogger()
}

private fun verifyLogDoesNotContain(regex: String) {
verify(externalLogger, never()).log(
any(),
any(),
argThat(RegexMatcher(regex)),
ArgumentMatchers.anyBoolean()
)
}

private fun verifyLogCouldContain(regex: String) {
verify(externalLogger, never()).log(
any(),
any(),
argThat(RegexMatcher(regex)), // allowList items are logged but the containsPII should be true.
eq(false)
)
}

class RegexMatcher(private val regex: String) : ArgumentMatcher<String> {
override fun matches(argument: String?): Boolean {
return regex.toRegex().containsMatchIn(argument ?: "")
}
}
}