Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RH2090378: Revert to disabling system security properties and FIPS mode support together #4

Merged
merged 9 commits into from Jun 22, 2022

Conversation

gnu-andrew
Copy link

@gnu-andrew gnu-andrew commented Jun 6, 2022

Search this PR in Red Hat Jira

Introduce security.useFIPSProviders as a security property equivalent of the system property com.redhat.fips. Improve debug output of all four properties for FIPS mode and system security property support.

Note that the security properties are both now turned off by default, so behaviour is initially like upstream. We will enable them in the RPM. Once both are turned on in the file, the behaviour is as shown below (TestProviders is a simple piece of code to list the security providers installed)

$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties TestProviders
properties: java.security.disableSystemPropertiesFile=false
properties: security.useSystemPropertiesFile=true
properties: reading system security properties file /etc/crypto-policies/back-ends/java.config
properties: {...}
properties: com.redhat.fips=true
properties: security.useFIPSProviders=true
properties: Calling getSystemFIPSEnabled (libsystemconf)...
properties: getSystemFIPSEnabled: calling SECMOD_GetSystemFIPSEnabled
properties: getSystemFIPSEnabled: SECMOD_GetSystemFIPSEnabled returned 0x0
properties: Call to getSystemFIPSEnabled (libsystemconf) returned: false
properties: FIPS mode not detected. Support disabled.
$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties -Djava.security.disableSystemPropertiesFile=true TestProviders
properties: java.security.disableSystemPropertiesFile=true
properties: security.useSystemPropertiesFile=true
properties: System security property support disabled by user.
properties: com.redhat.fips=true
properties: security.useFIPSProviders=true
properties: Calling getSystemFIPSEnabled (libsystemconf)...
properties: getSystemFIPSEnabled: calling SECMOD_GetSystemFIPSEnabled
properties: getSystemFIPSEnabled: SECMOD_GetSystemFIPSEnabled returned 0x0
properties: Call to getSystemFIPSEnabled (libsystemconf) returned: false
properties: FIPS mode not detected. Support disabled.
$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties -Djava.security.disableSystemPropertiesFile=true -Dcom.redhat.fips=false
properties: java.security.disableSystemPropertiesFile=true
properties: security.useSystemPropertiesFile=true
properties: System security property support disabled by user.
properties: com.redhat.fips=false
properties: security.useFIPSProviders=true
properties: FIPS mode support disabled by user.

If FIPS mode is detected but system security property support is off, the following is printed and FIPS mode support is disabled. This allows -Djava.security.disableSystemPropertiesFile=true or the java.security equivalent alone to turn it off as before.

properties: FIPS mode support can not be enabled without system security properties being enabled.
properties: Please set security.useSystemPropertiesFile to true.
properties: FIPS mode support disabled.

…system

Introduce security.useFIPSProviders as a security property equivalent of
the system property com.redhat.fips. Improve debug output of all four
properties for FIPS mode and system security property support.
Run JDK tests in GHA with system security properties and FIPS mode support enabled in java.security
…rmer always takes precedence.

Fix spacing in submit.yml
@gnu-andrew
Copy link
Author

Reworked the handling of the new property so it -Dcom.redhat.fips takes precedence and can be used to override what is in java.security.
With the default java.security file (both false):

$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties TestProviders
properties: com.redhat.fips=<unset>
properties: security.useFIPSProviders=false
properties: com.redhat.fips unset, using security.useFIPSProviders
properties: FIPS mode support disabled by user.
$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties -Dcom.redhat.fips=true TestProviders
properties: com.redhat.fips=true
properties: security.useFIPSProviders=false
properties: com.redhat.fips set, using its value true
properties: Calling getSystemFIPSEnabled (libsystemconf)...
properties: getSystemFIPSEnabled: calling SECMOD_GetSystemFIPSEnabled
properties: getSystemFIPSEnabled: SECMOD_GetSystemFIPSEnabled returned 0x0
properties: Call to getSystemFIPSEnabled (libsystemconf) returned: false
properties: FIPS mode not detected
properties: FIPS mode support disabled.
$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties -Dcom.redhat.fips=false TestProviders
properties: com.redhat.fips=false
properties: security.useFIPSProviders=false
properties: com.redhat.fips set, using its value false
properties: FIPS mode support disabled by user.

@gnu-andrew gnu-andrew changed the title RH2090378: Add an option that disables OpenJDK FIPS globally RH2090378: Revert to disabling system security properties and FIPS mode support together Jun 15, 2022
@martinuy
Copy link

We have looked in detail into the proposed set of changes with @franferrax. For convenience, here it's a diff with all changes squashed. We will separate our observations into 3 categories:

Semantic equivalence to the old behavior

There seems to be semantic equivalence, to the best of our understanding.

Visible changes

With the proposed changes, the method SystemConfigurator::configureFIPS is invoked even when the alignment with the system security properties (crypto-policies) is disabled. All that is needed is the system property com.redhat.fips not to be false. The function SystemConfigurator::configureFIPS will invoke SystemConfigurator::enableFips in all cases. The function SystemConfigurator::enableFips will incur into several performance costs including a JNI call to the libsystemconf library, loading the NSS library with dlopen, executing dlsym to find the address of the relevant API looked up and, eventually, reading the FIPS value from the kernel (if the previous was unavailable). In the aforementioned scenario, the benefit would only be logging a debug line informing the user what is already known: alignment to system properties is disabled (logged already in the debug output) and the system is in FIPS mode (part of the execution context). We think that it is not worth generating such penalty at OpenJDK startup to users who explicitly disable the system security properties alignment in their RPM builds. Portable builds would be affected by default. This penalty was not incurred before because the disablement of the system security properties alignment prevented any further FIPS checks.

At a smaller scale, there can also be savings if checking the FIPS property (com.redhat.fips) were done only for the case in which there is alignment to the system security properties.

Code readability and debug outputs

We think that the overall code readability and debug outputs are not necessarily an improvement over the previous state but can, in some instances, add issues. These are some of the improvement possibilities that we see:

Security.java
  • disableSystemProps can be inlined into Boolean.valueOf
  • propsDisabled and useProps names can be aligned to reflect that they are system and security properties affecting the same configuration, perhaps with prefixes sys and sec. In the case of propsDisabled would be good to invert the semantics, at the code level at least. We also suggest exploring the use of sun.security.util.SecurityProperties from OpenJDK upstream.
  • The loadedProps check in line 236 might be superfluous but, most importantly, the FIPS property check should take place after verifying that the system security properties alignment is enabled. This would better reflect the current semantics. The decoupling at the Security.java level and coupling at the SystemConfigurator.java level by means of a static private field is, in our view, a step backwards in code clarity.
  • The debug output associated to the FIPS property previous to the SystemConfigurator.configureFIPS call along with the line 264 is unnecessarily verbose and can be simplified into a single line.
SystemConfigurator.java
  • We suggest to remove the static private field systemSecPropsEnabled as mentioned before.
  • loadedProps name in configureFIPS does not reflect its purpose
  • The TODO in line 113 should not be implemented in our view because it's not OpenJDK's responsibility to check for system consistency. The assumption is that the user did not tamper any system configuration.
  • The comments previous to enableFips are outdated.

Finally, we think that the old/existing code should be improved but we have some concerns about the direction in which it's going with the current series of proposed changes. Ideally, we would like to redefine the set of requirements for the FIPS functionality and refactor the old code with that clear. In example, if decoupling FIPS from system security properties is the way forward at a functional level, the code should not be in a hybrid state (as proposed with the last change) but fully aligned to that. Conversely, if the way forward is not decoupling, the code should also reflect that.

@franferrax
Copy link

franferrax commented Jun 15, 2022

We have looked in detail into the proposed set of changes with @franferrax. For convenience, here it's a diff with all changes squashed.
[...]

How to see PR changes on top of a base commit (including it)

For our own future reference, this is how we generated 4ac1a03...afff64a.

rh_repo='https://github.com/rh-openjdk/jdk'
base_commit='2028344a5c354ac14b8c6ea4ca5a71622cd5e524'
pr_number=4
branch="PR4onTopOf${base_commit:0:7}"

# Get the fips-17u and this PR branches
git fetch ${rh_repo} fips-17u:fips-17u
git fetch ${rh_repo} pull/$pr_number/head:pr-branch

# Create a branch from the base commit
git switch -c ${branch} ${base_commit}

# Cherry pick the PR changes, in this case ignoring merges
git cherry-pick --no-merges fips-17u..pr-branch

# Push the branch to my GitHub fork
git push origin ${branch}

# Generate comparison URL: parent of ${base_commit} -> ${branch}
bold_green="$(tput setaf 2)$(tput bold)" && reset="$(tput sgr0)"
range="$(git rev-parse ${base_commit}^)...$(git rev-parse ${branch})"
echo "${bold_green}See changes in $rh_repo/compare/${range}${reset}"

# Cleanup local changes (keeping commits in GitHub for the record)
git switch master && git branch -D ${branch} pr-branch fips-17u
git reflog expire --expire-unreachable=now --all && git gc --prune=now

NOTE: as long as the franferrax:PR4onTopOf2028344 branch isn't deleted, the generated diff view will be persisted. If the branch is deleted, GitHub might garbage collect those dangling commits.

@gnu-andrew
Copy link
Author

We have looked in detail into the proposed set of changes with @franferrax. For convenience, here it's a diff with all changes squashed.
[...]

How to see PR changes on top of a base commit (including it)

For our own future reference, this is how we generated 4ac1a03...afff64a.

This isn't correct. It includes changes which are not part of this PR. You can see the changes in this PR by clicking the "Files Changed" tab or running git diff rh/fips-17u on the checked out branch, where rh is git@github.com:rh-openjdk/jdk.git

@gnu-andrew
Copy link
Author

We have looked in detail into the proposed set of changes with @franferrax. For convenience, here it's a diff with all changes squashed. We will separate our observations into 3 categories:

Semantic equivalence to the old behavior

There seems to be semantic equivalence, to the best of our understanding.

I think this is the key point; the goal of the patch is achieved.

Many of the suggestions below are worthwhile improvements, but would change the behaviour. I'm well aware that the result of this patch is not perfect, but the intention is to do a fairly minimal reversion, not to rename properties or change their behaviour.

Visible changes

With the proposed changes, the method SystemConfigurator::configureFIPS is invoked even when the alignment with the system security properties (crypto-policies) is disabled. All that is needed is the system property com.redhat.fips not to be false. The function SystemConfigurator::configureFIPS will invoke SystemConfigurator::enableFips in all cases. The function SystemConfigurator::enableFips will incur into several performance costs including a JNI call to the libsystemconf library, loading the NSS library with dlopen, executing dlsym to find the address of the relevant API looked up and, eventually, reading the FIPS value from the kernel (if the previous was unavailable). In the aforementioned scenario, the benefit would only be logging a debug line informing the user what is already known: alignment to system properties is disabled (logged already in the debug output) and the system is in FIPS mode (part of the execution context). We think that it is not worth generating such penalty at OpenJDK startup to users who explicitly disable the system security properties alignment in their RPM builds. Portable builds would be affected by default. This penalty was not incurred before because the disablement of the system security properties alignment prevented any further FIPS checks.

The intention was to be able to tell the user as much as possible (your system is in FIPS mode, but we can't turn on FIPS support), but re-evaluating this, I agree the performance cost isn't worth it. I'll move the check to wrap the entire FIPS configuration instead.

At a smaller scale, there can also be savings if checking the FIPS property (com.redhat.fips) were done only for the case in which there is alignment to the system security properties.

As above.

Code readability and debug outputs

We think that the overall code readability and debug outputs are not necessarily an improvement over the previous state but can, in some instances, add issues. These are some of the improvement possibilities that we see:

Security.java
* `disableSystemProps` can be inlined into `Boolean.valueOf`

The reason I avoided this was because the length of the properties makes the line length too long. I've changed them to constants to avoid this, which also allows them to be reused in the debugging output.

* `propsDisabled` and `useProps` names can be aligned to reflect that they are system and security properties affecting the same configuration, perhaps with prefixes _sys_ and _sec_. In the case of `propsDisabled` would be good to invert the semantics, at the code level at least. We also suggest exploring the use of `sun.security.util.SecurityProperties` from OpenJDK upstream.

I've renamed them as suggested. I think inverting the semantics in the code would create even more confusion. It's already hard enough for me to follow.

I looked briefly at sun.security.util.SecurityProperties but it would enforce the consistent naming which we don't yet support. Also, given the file is dated 2018, I doubt the class is available back to 8u, and I'd prefer not to have to take it out again in a backport.

* The `loadedProps` check in line 236 might be superfluous but, most importantly, the FIPS property check should take place after verifying that the system security properties alignment is enabled. This would better reflect the current semantics. The decoupling at the _Security.java_ level and coupling at the _SystemConfigurator.java_ level by means of a static private field is, in our view, a step backwards in code clarity.

The loadedProps check is not new to this patch. It is needed to make sure java.security has been loaded.

The FIPS property check is already after the system security properties alignment check.

You already said part of this above, I believe. I've moved the variable into Security.java.

* The debug output associated to the FIPS property previous to the `SystemConfigurator.configureFIPS` call along with the line 264 is unnecessarily verbose and can be simplified into a single line.

I've removed the first line. I guess it's a legacy of when there were originally two properties.

SystemConfigurator.java
* We suggest to remove the static private field `systemSecPropsEnabled` as mentioned before.

Done

* `loadedProps` name in `configureFIPS` does not reflect its purpose

Maybe, but this is not added or altered by this patch at all

* The TODO in line 113 should not be implemented in our view because it's not OpenJDK's responsibility to check for system consistency. The assumption is that the user did not tamper any system configuration.

I've taken it out, but I tend to disagree. If you're relying on the system security property system to limit the algorithms available, then you should really check that this is actually in place.

* The comments previous to `enableFips` are outdated.

Can you elaborate? I don't see how.

Finally, we think that the old/existing code should be improved but we have some concerns about the direction in which it's going with the current series of proposed changes. Ideally, we would like to redefine the set of requirements for the FIPS functionality and refactor the old code with that clear. In example, if decoupling FIPS from system security properties is the way forward at a functional level, the code should not be in a hybrid state (as proposed with the last change) but fully aligned to that. Conversely, if the way forward is not decoupling, the code should also reflect that.

This is definitely out of scope of this PR. I have no strong feelings either way as to whether FIPS depends on system security properties; that is an implementation choice for yourself and Francisco. It is certain that system security properties are used outside of FIPS mode and so should be entangled with that code as little as possible. As previously stated, I intend to shortly rework that code and take it upstream.

@gnu-andrew
Copy link
Author

Changes pushed. Please use the "Files changed" tab to review.

@gnu-andrew
Copy link
Author

Current output once property is set to true in java.security (as on RHEL):

$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties TestProviders
properties: reading security properties file: /home/andrew/builder/17u-fips/images/jdk/conf/security/java.security
properties: {...}
properties: java.security.disableSystemPropertiesFile=false
properties: security.useSystemPropertiesFile=true
properties: reading system security properties file /etc/crypto-policies/back-ends/java.config
properties: {..}
properties: com.redhat.fips unset, using default value of true
properties: Calling getSystemFIPSEnabled (libsystemconf)...
properties: getSystemFIPSEnabled: calling SECMOD_GetSystemFIPSEnabled
properties: getSystemFIPSEnabled: SECMOD_GetSystemFIPSEnabled returned 0x0
properties: Call to getSystemFIPSEnabled (libsystemconf) returned: false
properties: FIPS mode not detected
properties: FIPS mode support disabled.
$ ~/builder/17u-fips/images/jdk/bin/java -Djava.security.debug=properties -Djava.security.disableSystemPropertiesFile=true TestProviders
properties: reading security properties file: /home/andrew/builder/17u-fips/images/jdk/conf/security/java.security
properties: {...}
properties: java.security.disableSystemPropertiesFile=true
properties: security.useSystemPropertiesFile=true
properties: System security property support disabled by user.
properties: WARNING: FIPS mode support can not be enabled without system security properties being enabled.

@franferrax
Copy link

This isn't correct. It includes changes which are not part of this PR. You can see the changes in this PR by clicking the "Files Changed" tab or running git diff rh/fips-17u on the checked out branch, where rh is git@github.com:rh-openjdk/jdk.git

Correctness depends on the context, we intentionally included 2028344 as part of the diff, and this is the only additional change outside of the PR. We did it behind the following rationale:

  • We hadn't had a chance to review 2028344
  • This PR is about restoring behavior prior to 2028344
  • It was the easiest way to see the changes ensuring earlier behavior is kept
  • We are searching for the improvements against the earlier state (i.e. the advantages against doing git revert 2028344)

Updated diff view: 4ac1a03...5e92115

@gnu-andrew
Copy link
Author

This isn't correct. It includes changes which are not part of this PR. You can see the changes in this PR by clicking the "Files Changed" tab or running git diff rh/fips-17u on the checked out branch, where rh is git@github.com:rh-openjdk/jdk.git

Correctness depends on the context, we intentionally included 2028344 as part of the diff, and this is the only additional change outside of the PR. We did it behind the following rationale:

* We hadn't had a chance to review [2028344](https://github.com/rh-openjdk/jdk/commit/2028344a5c354ac14b8c6ea4ca5a71622cd5e524)

It's been reviewed back in January and been shipped. This PR is not the place to review it as it is not contained within this PR.

* This PR is about restoring behavior prior to [2028344](https://github.com/rh-openjdk/jdk/commit/2028344a5c354ac14b8c6ea4ca5a71622cd5e524)

* It was the easiest way to see the changes ensuring earlier behavior is kept

* We are searching for the improvements against the earlier state (i.e. the advantages against doing `git revert 2028344`)

Updated diff view: 4ac1a03...5e92115

If this is useful to you, then good. But I want to look at the changes I'm proposing to commit, not a bunch of other changes that are already committed as well. I don't find it remotely helpful, it's just confusing.

Copy link

@martinuy martinuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's move forward

@gnu-andrew
Copy link
Author

Ok, let's move forward

Thank you!

@gnu-andrew
Copy link
Author

JTREG="JAVA_OPTIONS='-Djava.security.debug=properties -XX:-CreateCoredumpOnCrash'"

I'm not sure, but it seems that the single quotes here are triggering a java invocation with -javaoption:'-Djava.security.debug=properties -javaoption:-XX:-CreateCoredumpOnCrash', causing some Unrecognized option: -javaoption:-XX:-CreateCoredumpOnCrash, making fail the Linux x64 + FIPS (jdk/tier1...) runs.

If I'm not misinterpreting Test suite control in jdk17u/doc/testing.md, we can just remove the single quotes, since keyword=value pairs are separated by semicolon.

I wondered about that, but the last run didn't seem to get to the tests, failing with some odd download error, so this is the first time I've seen the output.

The doc you referenced suggests we can unquote and use %20 as in JTREG=JAVA_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug

@gnu-andrew gnu-andrew merged commit f8142a2 into rh-openjdk:fips-17u Jun 22, 2022
@franferrax
Copy link

The doc you referenced suggests we can unquote and use %20 as in JTREG=JAVA_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug

Yes, I was going to mention that, but it also says "This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through". I don't think it describes our case, since it looks like the issue is caused by the inner quotation (single quotes), and we are using the outer quotation without issues (double quotes).

I would first try JTREG="JAVA_OPTIONS=-Djava.security.debug=properties -XX:-CreateCoredumpOnCrash" without %20, to also confirm if %20 is actually needed. But using it might save us a try, so you may prefer to go that way.

@gnu-andrew
Copy link
Author

JTREG="JAVA_OPTIONS='-Djava.security.debug=properties -XX:-CreateCoredumpOnCrash'"

I'm not sure, but it seems that the single quotes here are triggering a java invocation with -javaoption:'-Djava.security.debug=properties -javaoption:-XX:-CreateCoredumpOnCrash', causing some Unrecognized option: -javaoption:-XX:-CreateCoredumpOnCrash, making fail the Linux x64 + FIPS (jdk/tier1...) runs.
If I'm not misinterpreting Test suite control in jdk17u/doc/testing.md, we can just remove the single quotes, since keyword=value pairs are separated by semicolon.

I wondered about that, but the last run didn't seem to get to the tests, failing with some odd download error, so this is the first time I've seen the output.

The doc you referenced suggests we can unquote and use %20 as in JTREG=JAVA_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug

The doc you referenced suggests we can unquote and use %20 as in JTREG=JAVA_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug

Yes, I was going to mention that, but it also says "This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through". I don't think it describes our case, since it looks like the issue is caused by the inner quotation (single quotes), and we are using the outer quotation without issues (double quotes).

I would first try JTREG="JAVA_OPTIONS=-Djava.security.debug=properties -XX:-CreateCoredumpOnCrash" without %20, to also confirm if %20 is actually needed. But using it might save us a try, so you may prefer to go that way.

I've tried %20 in this PR - #11 - so as not to disturb the existing review on this one (now merged). I'll merge if the results look good. You're welcome to try with a space, though I'm not sure if the outer double quotes will be removed before being passed through.

gnu-andrew added a commit that referenced this pull request Jun 24, 2022
…de support together (#4)

- Improve debug output of all properties for FIPS mode and system security property support.
- Run JDK tests in GHA with system security properties both disabled and enabled in java.security
- General code cleanup

Reviewed-by: @martinuy
Reviewed-by: @franferrax
gnu-andrew added a commit that referenced this pull request Jul 13, 2022
…de support together (#4)

- Improve debug output of all properties for FIPS mode and system security property support.
- Run JDK tests in GHA with system security properties both disabled and enabled in java.security
- General code cleanup

Reviewed-by: @martinuy 
Reviewed-by: @franferrax
gnu-andrew added a commit that referenced this pull request Aug 28, 2022
…de support together (#4)

- Improve debug output of all properties for FIPS mode and system security property support.
- Run JDK tests in GHA with system security properties both disabled and enabled in java.security
- General code cleanup

Reviewed-by: @martinuy
Reviewed-by: @franferrax
gnu-andrew added a commit that referenced this pull request Apr 3, 2023
…de support together (#4)

- Improve debug output of all properties for FIPS mode and system security property support.
- Run JDK tests in GHA with system security properties both disabled and enabled in java.security
- General code cleanup

Reviewed-by: @martinuy
Reviewed-by: @franferrax
gnu-andrew added a commit that referenced this pull request Aug 22, 2023
…de support together (#4)

- Improve debug output of all properties for FIPS mode and system security property support.
- Run JDK tests in GHA with system security properties both disabled and enabled in java.security
- General code cleanup

Reviewed-by: @martinuy
Reviewed-by: @franferrax
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants