-
Notifications
You must be signed in to change notification settings - Fork 363
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #619 from kwwall/documentation-changes
Documentation changes
- Loading branch information
Showing
11 changed files
with
327 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| (Gleaned from an email from Jeremiah J. Stacey to Kevin W. Wall on 2021-03-21. Some minor alterations were made for contextual understanding by Kevin Wall because the original email thread was not included.) | ||
|
|
||
| The testing (for SLF4J logging at least) is tested with Mockito and Powermock. For the SLF4J logging, the tests are in Slf4JLoggerTest. It uses mocks to assert that the slf4j logging implementation gets the data we expect in the calls we support. | ||
|
|
||
| I was very deliberate in the breakout of the classes to isolate specific functionality to enable this type of testing. At a high level, there are four classes that make up the logging structure. | ||
| I tried to encapsulate a subset of functionality into each one: | ||
|
|
||
| **LogFactory** - Constructs Loggers to be used by clients. Responsible for building the LogBridge and the LevelHandler. | ||
|
|
||
| **Logger** - The ESAPI interface implementation which uses the LogBridge and a delegate Logger to forward events to the underlying log implementation. | ||
|
|
||
| **LogBridge** - Logical handler for determining the delegate handler for a known ESAPI log event, and forwarding the Log event to that handler. Also responsible for prefixing the client/server info content and applying the newline replacement behavior. | ||
|
|
||
| **LogLevelHander** - This is actually where the log event gets sent to SLF4J! The Handler enumeration is assembled as part of a map in the static block of the LogFactory, and is used by the LogBridge to route a log event at a defined ESAPI log level to the correct API of the delegate Logger. | ||
|
|
||
|
|
||
| The general workflow is: | ||
|
|
||
| LogFactory static block creates the LogPrefixAppender, LogScrubber, and LogBridge. | ||
|
|
||
| LogFactory.getLogger(...) Creates Logger with the delegate slf4j logger implementation and the LogBridge. | ||
|
|
||
| Logger.info/warn/etc(message) -> forwards to LogBridgelog(logger, esapiLevel, type, message) -> forwards to LogHandler.log(...) -> forwards to slf4j Logger implementation with appropriate level and composed message. | ||
|
|
||
| So each of the tests for each of the classes verifies data in -> data out based on the Logging API. The structure for JUL, Log4J, and SLF4J are almost identical. There are a few differences in the interaction with the underlying Logger interactions and expectations. As a result, the tests are also almost full duplications (again accounting for differences in the underlying logging API). | ||
|
|
||
| -J |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| #!/bin/bash | ||
| # Purpose: Answer some questions and provide a new 'vars.<version>' from 'vars.template' to use for creating release notes. | ||
|
|
||
| prog="${0##*/}" | ||
|
|
||
| function iprompt # prompt_message | ||
| { | ||
| typeset ANS | ||
| read -p "$@ (y|n): " ANS | ||
| case "$ANS" in | ||
| [Yy]|[Yy][Ee][Ss]) return 0 ;; | ||
| *) return 1 ;; | ||
| esac | ||
| } | ||
|
|
||
| read -p "Enter release # for NEW ESAPI version you are doing release notes for: " VERSION | ||
| if [[ -f "vars.$VERSION" ]] | ||
| then | ||
| iprompt "File 'vars.$VERSION' already exists. Continuing will overwrite it. Continue?" || exit 1 | ||
| fi | ||
|
|
||
|
|
||
| read -p "Enter release # for the PREVIOUS ESAPI version: " PREV_VERSION | ||
| read -p "Enter (planned) release date of NEW / current version you are preparing in YYYY-MM-DD format: " YYYY_MM_DD_RELEASE_DATE | ||
| read -p "Enter release date of PREVIOUS ESAPI version in YYYY-MM-DD format: " PREV_RELEASE_DATE | ||
|
|
||
| echo You entered: | ||
| echo ================================================= | ||
| echo VERSION=$VERSION | ||
| echo PREV_VERSION=$PREV_VERSION | ||
| echo YYYY_MM_DD_RELEASE_DATE=$YYYY_MM_DD_RELEASE_DATE | ||
| echo PREV_RELEASE_DATE=$PREV_RELEASE_DATE | ||
| echo ================================================= | ||
| echo | ||
|
|
||
| if iprompt "Are ALL your previous answers correct?" | ||
| then | ||
| # Create the new vars.${VERSION} file based on vars.template | ||
| sed -e "s/^VERSION/VERSION=$VERSION/" \ | ||
| -e "s/^PREV_VERSION/PREV_VERSION=$PREV_VERSION/" \ | ||
| -e "s/^YYYY_MM_DD_RELEASE_DATE/YYYY_MM_DD_RELEASE_DATE=$YYYY_MM_DD_RELEASE_DATE/" \ | ||
| -e "s/^PREV_RELEASE_DATE/PREV_RELEASE_DATE=$PREV_RELEASE_DATE/" \ | ||
| vars.template > "vars.$VERSION" | ||
| else | ||
| echo "$prog: Aborting. Rerun the script to correct your answers." >&2 | ||
| exit 1 | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| @@@@ IMPORTANT: Be sure to 1) save in DOS text format, and 2) Delete this line and others starting with @@@@ | ||
| @@@ Edit with :set tw=0 | ||
| @@@@ Meant to be used with scripts/newReleaseNotes.sh and the 'vars.*' scripts there. | ||
| Release notes for ESAPI ${VERSION} | ||
| Release date: ${YYYY_MM_DD_RELEASE_DATE} | ||
| Project leaders: | ||
| -Kevin W. Wall <kevin.w.wall@gmail.com> | ||
| -Matt Seil <matt.seil@owasp.org> | ||
|
|
||
| Previous release: ESAPI ${PREV_VERSION}, ${PREV_RELEASE_DATE} | ||
|
|
||
|
|
||
| Executive Summary: Important Things to Note for this Release | ||
| ------------------------------------------------------------ | ||
| @@@@ View previous release notes to see examples of what to put here. This is typical. YMMV. | ||
| This is a patch release with the primary intent of updating some dependencies, some with known vulnerabilities. Details follow. | ||
|
|
||
| For those of you using a Software Configuration Analysis (SCA) services such as Snyk, BlackDuck, Veracode SourceClear, OWASP Dependency Check, etc., you might notice that there is vulnerability in xerces:xercesImpl:2.12.0 that ESAPI uses (also a transitive dependency) that is similar to CVE-2020-14621. Unfortunately there is no official patch for this in the regular Maven Central repository. Further details are described in Security Bulletin #3, which is viewable here | ||
| https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-security-bulletin3.pdf | ||
| and associated with this release on GitHub. Manual workarounds possible. See the security bulletin for further details. | ||
|
|
||
|
|
||
| ================================================================================================================= | ||
|
|
||
| Basic ESAPI facts | ||
| ----------------- | ||
|
|
||
| ESAPI ${PREV_VERSION} release: | ||
| @@@@ Look up the figures from the previous release notes. | ||
| #### Java source files | ||
| #### JUnit tests in #### Java test files | ||
|
|
||
| ESAPI ${version} release: | ||
| @@@@ Count them and run 'mvn test' to get the # of tests. | ||
| #### Java source files | ||
| #### JUnit tests in #### Java source files | ||
|
|
||
| XXX GitHub Issues closed in this release, including those we've decided not to fix (marked '(wontfix)'). | ||
| (Reference: https://github.com/ESAPI/esapi-java-legacy/issues?q=is%3Aissue+state%3Aclosed+updated%3A%3E%3D${PREV_RELEASE_DATE}) | ||
|
|
||
| Issue # GitHub Issue Title | ||
| ---------------------------------------------------------------------------------------------- | ||
| @@@@ Capture issue #s and 1 line desription from above GitHub url | ||
| @@@@ Insert here and massage until it looks pretty. Recommend alignment with spaces instead of tabs. | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Changes Requiring Special Attention | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
| @@@@ NOTE any special notes here. Probably leave this one, but I would suggest noting additions BEFORE this. | ||
| [If you have already successfully been using ESAPI 2.2.1.0 or later, you probably can skip this section.] | ||
|
|
||
| Since ESAPI 2.2.1.0, the new default ESAPI logger is JUL (java.util.logging packages) and we have deprecated the use of Log4J 1.x because we now support SLF4J and Log4J 1.x is way past its end-of-life. We did not want to make SLF4J the default logger (at least not yet) as we did not want to have the default ESAPI use require additional dependencies. However, SLF4J is likely to be the future choice, at least once we start on ESAPI 3.0. A special shout-out to Jeremiah Stacey for making this possible by re-factoring much of the ESAPI logger code. Note, the straw that broke the proverbial camel's back was the announcement of CVE-2019-17571 (rated Critical), for which there is no fix available and likely will never be. | ||
|
|
||
| However, if you try to juse the new ESAPI 2.2.1.0 or later logging you will notice that you need to change ESAPI.Logger and also possibly provide some other properties as well to get the logging behavior that you desire. | ||
|
|
||
| To use ESAPI logging in ESAPI 2.2.1.0 (and later), you will need to set the ESAPI.Logger property to | ||
|
|
||
| org.owasp.esapi.logging.java.JavaLogFactory - To use the new default, java.util.logging (JUL) | ||
| org.owasp.esapi.logging.log4j.Log4JLogFactory - To use the end-of-life Log4J 1.x logger | ||
| org.owasp.esapi.logging.slf4j.Slf4JLogFactory - To use the new (to release 2.2.0.0) SLF4J logger | ||
|
|
||
| In addition, if you wish to use JUL for logging, you *MUST* supply an "esapi-java-logging.properties" file in your classpath. This file is included in the 'esapi-2.2.2.0-configuration.jar' file provided under the 'Assets' section of the GitHub Release at | ||
| https://github.com/ESAPI/esapi-java-legacy/releases/esapi-2.2.2.0 | ||
|
|
||
| Unfortunately, there was a logic error in the static initializer of JavaLogFactory (now fixed in this release) that caused a NullPointerException to be thrown so that the message about the missing "esapi-java-logging.properties" file was never seen. | ||
|
|
||
| If you are using JavaLogFactory, you will also want to ensure that you have the following ESAPI logging properties set: | ||
| # Set the application name if these logs are combined with other applications | ||
| Logger.ApplicationName=ExampleApplication | ||
| # If you use an HTML log viewer that does not properly HTML escape log data, you can set LogEncodingRequired to true | ||
| Logger.LogEncodingRequired=false | ||
| # Determines whether ESAPI should log the application name. This might be clutter in some single-server/single-app environments. | ||
| Logger.LogApplicationName=true | ||
| # Determines whether ESAPI should log the server IP and port. This might be clutter in some single-server environments. | ||
| Logger.LogServerIP=true | ||
| # LogFileName, the name of the logging file. Provide a full directory path (e.g., C:\\ESAPI\\ESAPI_logging_file) if you | ||
| # want to place it in a specific directory. | ||
| Logger.LogFileName=ESAPI_logging_file | ||
| # MaxLogFileSize, the max size (in bytes) of a single log file before it cuts over to a new one (default is 10,000,000) | ||
| Logger.MaxLogFileSize=10000000 | ||
| # Determines whether ESAPI should log the user info. | ||
| Logger.UserInfo=true | ||
| # Determines whether ESAPI should log the session id and client IP. | ||
| Logger.ClientInfo=true | ||
|
|
||
| See GitHub issue #560 for additional details. | ||
|
|
||
|
|
||
| Related to that aforemented Log4J 1.x CVE and how it affects ESAPI, be sure to read | ||
| https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-security-bulletin2.pdf | ||
| which describes CVE-2019-17571, a deserialization vulnerability in Log4J 1.2.17. ESAPI is *NOT* affected by this (even if you chose to use Log4J 1 as you default ESAPI logger). This security bulletin describes why this CVE is not exploitable as used by ESAPI. | ||
|
|
||
|
|
||
| Finally, while ESAPI still supports JDK 7 (even though that too is way past end-of-life), the next ESAPI release will move to JDK 8 as the minimal baseline. (We already use Java 8 for development but still to Java 7 source and runtime compatibility.) We need to do this out of necessity because some of our dependencies are no longer doing updates that support Java 7. | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Remaining Known Issues / Problems | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
| If you use Java 7 (the minimal Java baseline supported by ESAPI) and try to run 'mvn test' there is one test that fails. This test passes with Java 8. The failing test is: | ||
|
|
||
| [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.203 s | ||
| <<< FAILURE! - in org.owasp.esapi.crypto.SecurityProviderLoaderTest | ||
| [ERROR] org.owasp.esapi.crypto.SecurityProviderLoaderTest.testWithBouncyCastle | ||
| Time elapsed: 0.116 s <<< FAILURE! | ||
| java.lang.AssertionError: Encryption w/ Bouncy Castle failed with | ||
| EncryptionException for preferred cipher transformation; exception was: | ||
| org.owasp.esapi.errors.EncryptionException: Encryption failure (unavailable | ||
| cipher requested) | ||
| at | ||
| org.owasp.esapi.crypto.SecurityProviderLoaderTest.testWithBouncyCastle(Security | ||
| ProviderLoaderTest.java:133) | ||
|
|
||
| I will spare you all the details and tell you that this has to do with Java 7 not being able to correctly parse the signed Bouncy Castle JCE provider jar. More details are available at: | ||
| https://www.bouncycastle.org/latest_releases.html | ||
| and | ||
| https://github.com/bcgit/bc-java/issues/477 | ||
| I am sure that there are ways of making Bouncy Castle work with Java 7, but since ESAPI does not rely on Bouncy Castle (it can use any compliant JCE provider), this should not be a problem. (It works fine with the default SunJCE provider.) If it is important to get the BC provider working with the ESAPI Encryptor and Java 7, then open a GitHub issue and we will take a deeper look at it and see if we can suggest something. | ||
|
|
||
|
|
||
|
|
||
| Another problem is if you run 'mvn test' from the 'cmd' prompt (and possibly PowerShell as well), you will get intermittent failures (generally between 10-25% of the time) at arbitrary spots. If you run it again without any changes it will work fine without any failures. We have discovered that it doesn't seem to fail if you run the tests from an IDE like Eclipse or if you redirect both stdout and stderr to a file; e.g., | ||
|
|
||
| C:\code\esapi-java-legacy> mvn test >testoutput.txt 2>&1 | ||
|
|
||
| We believe these failures is because the maven-surefire-plugin is by default not forking a new JVM process for each test class. We are looking into this. For now, we have only have observed this behavior on Windows 10. If you see this error, please do NOT report it as a GitHub issue unless you know a fix for it. (And yes, we are aware of '<reuseForks>false</reuseForks>' in the pom for the maven-surefire-plugin, but that causes other tests to fail that we haven't had time to fix.) | ||
|
|
||
|
|
||
| Lastly, some SCA services may continue to flag vulnerabilties in ESAPI ${VERSION} related to log4j 1.2.17 (e.g., CVE-2020-9488). We do not believe the way that ESAPI uses log4j in a manner that leads to any exploitable behavior. See the security bulletins | ||
| https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-security-bulletin2.pdf | ||
| for additional details. | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Other changes in this release, some of which not tracked via GitHub issues | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| * Minor updates to README.md file | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Developer Activity Report (Changes between release ${PREV_VERSION} and ${VERSION}, i.e., between ${PREV_RELEASE_DATE} and ${YYYY_MM_DD_RELEASE_DATE}) | ||
| Generated manually (this time) -- all errors are the fault of kwwall and his inability to do simple arithmetic. | ||
|
|
||
| Developer Total Total Number # Merged | ||
| (GitHub ID) commits of Files Changed PRs | ||
| ======================================================== | ||
| jeremiahjstacey 8 6 1 | ||
| dependabot 1 1 1 | ||
| kwwall 7 8 0 | ||
| ======================================================== | ||
| Total PRs: 2 | ||
|
|
||
| There were also several snyk-bot PRs that were rejected for various reasons, mostly because 1) I was already making the proposed changes and preferred to do them in single commit or 2) there were other reasons for rejecting them (such as the dependency requiring Java 8). The proposed changes that were not outright rejected were included as part of commit a8a79bc5196653500ce664b7b063284e60bddaa0. | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| CHANGELOG: Create your own. May I suggest: | ||
|
|
||
| git log --stat --since=${PREV_RELEASE_DATE} --reverse --pretty=medium | ||
|
|
||
| which will show all the commits since just after the previous (${PREV_VERSION}) release. | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Direct and Transitive Runtime and Test Dependencies: | ||
|
|
||
| $ mvn dependency:tree | ||
| @@@@ Include output from 'mvn dependency:tree' here | ||
|
|
||
| ----------------------------------------------------------------------------- | ||
|
|
||
| Acknowledgments: | ||
| Another hat tip to Dave Wichers for promptly releasing AntiSamy 1.6.1. And thanks to Matt Seil, Jeremiah Stacey, and all the ESAPI users who make this worthwhile. This is for you. | ||
|
|
||
| A special thanks to the ESAPI community from the ESAPI project co-leaders: | ||
| Kevin W. Wall (kwwall) <== The irresponsible party for these release notes! | ||
| Matt Seil (xeno6696) |
Oops, something went wrong.