[ELY-2320] Add integrity support to FileSystemSecurityRealm #1709

Ashpan · 2022-05-12T18:46:53Z

http://issues.redhat.com/browse/ELY-2345
https://issues.redhat.com/browse/ELY-2320

Ashpan · 2022-05-13T14:54:48Z

@fjuma @Skyllarr @OndrejKotek
The original Elytron PR for this integrity RFE got into a weird state and ended up getting closed.
I've made changes to address all comments from the original PR

fjuma · 2022-06-03T18:31:57Z

auth/realm/base/pom.xml

@@ -85,6 +85,11 @@
            <groupId>org.wildfly.common</groupId>
            <artifactId>wildfly-common</artifactId>
        </dependency>
+        <dependency>
+            <groupId>org.apache.santuario</groupId>


Since there was some discussion about this one, it would be good to add a comment here to explain the reason that this dependency is being used.

fjuma · 2022-06-03T18:49:27Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/ElytronMessages.java

+    IllegalArgumentException invalidKeyPairArgument(String s);
+
+    @Message(id = 13011, value = "Unable to access master index file: %s")
+    IllegalStateException unableToAccessMainIndex(String s);


Is this used by anything now?

fjuma · 2022-06-03T18:49:38Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/ElytronMessages.java

+
+    @LogMessage(level = Logger.Level.TRACE)
+    @Message(id = 13012, value = "Unable to write checksum to main index")
+    void unableToWriteToMainIndex();


fjuma · 2022-06-03T18:54:47Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

     */
-    public FileSystemSecurityRealm(final Path root, final NameRewriter nameRewriter, final int levels, final boolean encoded, final Encoding hashEncoding, final Charset hashCharset, final Supplier<Provider[]> providers, final SecretKey secretKey) {
+    public FileSystemSecurityRealm(final Path root, final NameRewriter nameRewriter, final int levels, final boolean encoded, final Encoding hashEncoding, final Charset hashCharset, Supplier<Provider[]> providers, final SecretKey secretKey, final PrivateKey privateKey, final PublicKey publicKey) {


Any reason to remove final from the providers parameter?

fjuma · 2022-06-03T18:56:07Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

@@ -252,10 +317,10 @@ public FileSystemSecurityRealm(final Path root, final NameRewriter nameRewriter,
     * @param nameRewriter the name rewriter to apply to looked up names
     * @param levels the number of levels of directory hashing to apply
     * @param hashEncoding the string format for hashed passwords. Uses Base64 by default.
-     * @param hashCharset the character set to use when converting password strings to a byte array. Uses UTF-8 by default and must not be {@code null}.
+     * @param hashCharset  the character set to use when converting password strings to a byte array. Uses UTF-8 by default and must not be {@code null}.


Minor but looks like extra whitespace was introduced here.

fjuma · 2022-06-03T19:47:00Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/IntegrityException.java

@@ -0,0 +1,71 @@
+/*
+ * JBoss, Home of Professional Open Source.
+ * Copyright 2020 Red Hat, Inc., and individual contributors


fjuma · 2022-06-03T19:48:55Z

auth/realm/base/src/main/resources/schema/elytron-identity-1_2.xsd

@@ -0,0 +1,150 @@
+<?xml version="1.0" encoding="UTF-8"?>


When adding a new schema file, it's good to first copy/paste the existing file and just update the version and include this in a commit. Then introduce the new element(s) in a follow-up commit. This makes it easier to see the specific changes introduced. We also tend to create a separate ELY issue to track adding the new client schema version.

fjuma · 2022-06-03T20:27:46Z

auth/realm/base/src/main/resources/schema/elytron-identity-1_2.xsd

+        <xsd:all minOccurs="1" maxOccurs="1">
+            <xsd:element name="principal" type="name-type" minOccurs="1" maxOccurs="1"/>
+            <xsd:element name="credentials" type="credentials-type" minOccurs="0" maxOccurs="1"/>
+            <xsd:element name="attributes" type="attributes-type" minOccurs="0" maxOccurs="1"/>


The signature element should be added here too.

There's an example here that shows how to import and make use of an element from another schema:

https://github.com/wildfly/wildfly-core/blob/7b9da5eced8a006f827769ede7c5847a8290370b/server/src/main/resources/schema/wildfly-config_18_0.xsd#L274

fjuma · 2022-06-03T20:28:00Z

base/src/main/java/org/wildfly/security/WildFlyElytronBaseProvider.java

@@ -26,7 +26,6 @@
 import java.lang.reflect.Constructor;
 import java.security.NoSuchAlgorithmException;
 import java.security.Provider;
-import java.security.Provider.Service;


Why is this being removed?

It wasn't in use, although it's unrelated to my PR, I can revert this change

fjuma · 2022-06-03T20:28:34Z

tests/base/src/test/resources/filesystem-realm-exists/u/s/e/user-OVZWK4Q.xml

+    <credentials>
+        <password algorithm="clear" format="base64">AXNlY3JldFBhc3N3b3Jk</password>
+    </credentials>
+    <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">


Just thinking about the formatting issue we just discussed. Is this formatting valid here?

It isn't but this tests is checking an invalid signature. I thought it would be good for someone to be able to visualise the signature tag a bit better.
Should I set it to what the realm normally generates instead?

Is the signature only invalid due to the formatting or due to the actual content too?

The content is also invalid, seen here

wildfly-elytron/tests/base/src/test/resources/filesystem-realm-exists/u/s/e/user-OVZWK4Q.xml

Line 18 in c4f7abc

<SignatureValue>INVALID_SIGNATURE</SignatureValue>

Ashpan · 2022-06-07T14:17:03Z

@fjuma I've addressed all comments

Skyllarr · 2022-06-08T12:18:34Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

+                .setLevels(1)
+                .setProviders(ELYTRON_PASSWORD_PROVIDERS);
+        if (mode.equals("encryption") || mode.equals("both")) { securityRealmBuilder.setSecretKey(secretKey); }
+        else if (mode.equals("integrity") || mode.equals("both")) { securityRealmBuilder.setPrivateKey(privateKey); securityRealmBuilder.setPublicKey(publicKey); }


@Ashpan Just a minor, the else should be removed here. Now if mode.equals("both") is true, the private and public key are not set

Ah good catch

Skyllarr · 2022-06-08T13:02:39Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

+        while (realmIterator.hasNext()) {
+            Identity identity = (Identity) realmIterator.next();
+            if(! identity.verifyIntegrity()) {
+                identity.dispose();


@fjuma @Ashpan Do we want to log which identity failed the integrity check?

Yes that's a good point.

Theres a trace log here that runs whenever we verify any identity, should I still add another one here?

wildfly-elytron/auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

Line 1500 in 7e09a5d

ElytronMessages.log.tracef("FileSystemSecurityRealm - Error during verification. Signature for credential [%s] failed", name);

@Ashpan Ah yes, sorry I missed that. Although it could still happen that the validatePrincipalName(path.toString(), name) method fails instead of the digital signature method, and then no identity is logged right? So we can add a log to the validatePrincipalName method also and we know the identity is logged then

Ah thats true, I'll add another log in validatePrincipalName()

fjuma · 2022-06-09T16:59:54Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

-    @Test
-    public void testCreateIdentityWithLevelsEncryption() throws Exception {
+   @Test
+    public void testCreateIdentityWithLevelsIntegrity() throws Exception {


Just to check because it's a bit hard to tell from the diff, was the encryption test removed?

It doesn't look like the tests validate the signature that's written to the identity file. That would be good to add.

I think the encryption test got moved around, Ive put them all together now

fjuma · 2022-06-09T17:08:30Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

        newIdentity.dispose();

-        ModifiableRealmIdentity existingIdentity = securityRealm.getRealmIdentityForUpdate(new NamePrincipal("p"));
+        securityRealm = FileSystemSecurityRealm.builder()


There's no need to recreate the security realm, the existing instance can be used.

fjuma · 2022-06-09T17:11:04Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

+                .setPrivateKey(privateKey)
+                .setPublicKey(publicKey)
+                .setProviders(ELYTRON_PASSWORD_PROVIDERS)
+                .build();


Just a general comment. Since the same builders are being used in multiple places, might be better to introduce methods that return the two common builders.

fjuma · 2022-06-09T17:15:54Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

+                .setRoot(getRootPath())
+                .setLevels(3)
+                .setProviders(ELYTRON_PASSWORD_PROVIDERS);
+        if (mode.equals("encryption")) { securityRealmBuilder.setSecretKey(secretKey); }


Instead of this approach, it would be better to add secretKey, privateKey, and publicKey parameters to createAndLoadAndDeleteIdentity and then just set the builder methods if the values that are passed in aren't null.

fjuma · 2022-06-09T17:19:55Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

+        KeyPair pair = keyPairGen.generateKeyPair();
+        PrivateKey newPrivateKey = pair.getPrivate();
+        PublicKey newPublicKey = pair.getPublic();
+        securityRealm = FileSystemSecurityRealm.builder()


No need to recreate the security realm.

fjuma · 2022-06-09T17:20:16Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

+        newIdentity.setCredentials(Collections.singleton(new PasswordCredential(credential)));
+        newIdentity.dispose();
+
+        securityRealm = FileSystemSecurityRealm.builder()


fjuma · 2022-06-09T17:21:49Z

tests/base/src/test/java/org/wildfly/security/auth/FileSystemSecurityRealmTest.java

@@ -1089,4 +1348,22 @@ private Path getRootPath() throws Exception {
        return getRootPath(true);
    }



Just a general comment. Have you tested the case where the identity file has been tampered with?

Have you tested the case where the signature can't be written to the file for some reason and then verified that the file remains unchanged?

Just a general comment. Have you tested the case where the identity file has been tampered with?

Added 2 tests where identity is tampered.

Have you tested the case where the signature can't be written to the file for some reason and then verified that the file remains unchanged?

I've tested manually, I'm unable to make a test to get the code into a state where only during writing the signature does it fail. If I try to make that happen, it gets caught in the "verify signature" method instead of the unable to write signature portion, but I am now using the temporary file to make changes before copying that to the final file.

fjuma · 2022-06-14T20:56:43Z

@Ashpan Please also link ELY-2345 in the description here (that makes it easier for us to resolve the issues once approved). Thanks.

fjuma · 2022-06-14T21:29:44Z

auth/realm/base/src/main/resources/schema/elytron-identity-1_2.xsd

            <xsd:element name="credentials" type="credentials-type" minOccurs="0" maxOccurs="1"/>
            <xsd:element name="attributes" type="attributes-type" minOccurs="0" maxOccurs="1"/>
+            <xsd:element name="Signature" />


A type attribute should be added here that references the SignatureType from the xml dsig schema.

Should set minOccurs and maxOccurs too.

fjuma · 2022-06-14T21:38:57Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

+     * Re-generate the signatures for all the identities in this realm.
+     * This method is intended to be called after updating the key pair used by this realm.
+     *
+     * @throws RealmUnavailableException Thrown if unable to obtain the realm identity iterator or dispose and close the realm


I think we could reword this to something like:

@throws RealmUnavailableException if the realm is not able to handle requests for any reason

fjuma · 2022-06-15T21:09:07Z

auth/realm/base/src/main/resources/schema/elytron-identity-1_2.xsd

            <xsd:element name="credentials" type="credentials-type" minOccurs="0" maxOccurs="1"/>
            <xsd:element name="attributes" type="attributes-type" minOccurs="0" maxOccurs="1"/>
+            <xsd:element name="Signature" type="SignatureType" minOccurs="1" maxOccurs="1"/>


I think this needs to be type="xmldsig:SignatureType"

Shouldn't minOccurs be 0 since it's possible to have unencrypted files?

I think this needs to be type="xmldsig:SignatureType"

Updated

Shouldn't minOccurs be 0 since it's possible to have unencrypted files?

Ah good catch, updated

fjuma · 2022-06-15T21:31:04Z

auth/realm/base/src/main/resources/schema/elytron-identity-1_2.xsd


    <xsd:complexType name="identity-type">
        <xsd:all minOccurs="1" maxOccurs="1">
+            <xsd:element name="principal" type="name-type" minOccurs="1" maxOccurs="1"/>


Since this element is only added if integrity is enabled, minOccurs should be 0 here too.

fjuma · 2022-06-15T21:46:11Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

                    // Mixed versions unsupported.
                    throw ElytronMessages.log.fileSystemRealmInvalidContent(path, streamReader.getLocation().getLineNumber(), name);
                }
+
+                if (version.isAtLeast(Version.VERSION_1_2) && "principal".equals(streamReader.getLocalName())) {
+                    consumeContent(streamReader);


I think we need to be a bit careful here. We need to make sure that if the principal element is encountered and the version isn't at least 1.2, then the invalid content (i.e., fileSystemRealmInvalidContent) should be thrown.

Added error if version mismatch

Thanks! Just to simplify the logic and make it a bit easier to understand what's happening, I think you could update it to:

if ("principal".equals(streamReader.getLocalName()) { if (version.isAtLeast(...)) { ... } else { ... } }

fjuma · 2022-07-04T17:43:49Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

+
+    public static class IntegrityResult {
+        private final boolean valid;
+        private final ArrayList<Identity> identities;


Just wondering if this could be an ArrayList<String> instead? That way, you'd just build up the list while verifying integrity and wouldn't need to iterate over the identities in getIdentityNames.

I was just thinking if in the future this was to be used for another purpose and another property of the identity was required, it would be useful. Although I could change it to ArrayList<String> identityNames for this use case, and if the identities are required in the future, the identities arraylist could be added back

Yeah, I think we just need the string names for now.

fjuma · 2022-07-04T17:46:50Z

auth/realm/base/src/main/java/org/wildfly/security/auth/realm/FileSystemSecurityRealm.java

@@ -1358,4 +1654,26 @@ public XMLStreamWriter getXmlStreamWriter() {
            return xmlStreamWriter;
        }
    }
+
+    public static class IntegrityResult {


Javadoc should be added for this class and its methods.

Ashpan force-pushed the ELY-2320 branch 4 times, most recently from 78476ad to 930e194 Compare May 17, 2022 15:31

Ashpan force-pushed the ELY-2320 branch 4 times, most recently from 7fb7e33 to 5bb9d35 Compare June 3, 2022 16:22

fjuma requested changes Jun 3, 2022

View reviewed changes

Ashpan force-pushed the ELY-2320 branch 5 times, most recently from 117ae93 to 6a52254 Compare June 7, 2022 14:16

Ashpan force-pushed the ELY-2320 branch 2 times, most recently from 23cfe36 to 7e09a5d Compare June 7, 2022 18:20

Skyllarr reviewed Jun 8, 2022

View reviewed changes

Ashpan force-pushed the ELY-2320 branch from 7e09a5d to 9049117 Compare June 8, 2022 17:15

fjuma reviewed Jun 9, 2022

View reviewed changes

Ashpan mentioned this pull request Jun 9, 2022

[WFCORE-5859] Add Filesystem integrity support wildfly/wildfly-core#5048

Merged

Ashpan force-pushed the ELY-2320 branch from 9049117 to 1f86e7b Compare June 14, 2022 15:16

fjuma reviewed Jun 14, 2022

View reviewed changes

Ashpan force-pushed the ELY-2320 branch from 1f86e7b to dc49f0c Compare June 15, 2022 19:14

fjuma reviewed Jun 15, 2022

View reviewed changes

Ashpan force-pushed the ELY-2320 branch 3 times, most recently from 893200a to 0650738 Compare June 17, 2022 13:30

Ashpan mentioned this pull request Jun 17, 2022

Blog post for filesystem integrity #1716

Closed

Ashpan force-pushed the ELY-2320 branch from 0650738 to 2621f0f Compare July 4, 2022 17:01

fjuma reviewed Jul 4, 2022

View reviewed changes

Ashpan force-pushed the ELY-2320 branch from 2621f0f to ef968fa Compare July 5, 2022 16:10

Ashpan force-pushed the ELY-2320 branch from ef968fa to fa14e79 Compare July 14, 2022 15:24

Ashpan added 2 commits July 26, 2022 10:16

[ELY-2345] New elytron-identity client schema version (1.2)

20504d9

[ELY-2320] Add integrity support to FileSystemSecurityRealm

8545a08

Ashpan force-pushed the ELY-2320 branch from fa14e79 to 8545a08 Compare July 26, 2022 14:16

fjuma approved these changes Jul 27, 2022

View reviewed changes

Skyllarr approved these changes Jul 27, 2022

View reviewed changes

fjuma merged commit ac53e1c into wildfly-security:1.x Jul 27, 2022

cam-rod mentioned this pull request Nov 10, 2022

Blog post for filesystem integrity #1841

Merged

		@@ -1089,4 +1348,22 @@ private Path getRootPath() throws Exception {
		return getRootPath(true);
		}

[ELY-2320] Add integrity support to FileSystemSecurityRealm #1709

[ELY-2320] Add integrity support to FileSystemSecurityRealm #1709

Conversation

Ashpan commented May 12, 2022 • edited

Ashpan commented May 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ashpan commented Jun 7, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Skyllarr Jun 8, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjuma commented Jun 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjuma Jun 16, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ashpan commented May 12, 2022 •

edited

Ashpan commented May 13, 2022 •

edited

Skyllarr Jun 8, 2022 •

edited

fjuma Jun 16, 2022 •

edited