NIFI-10167: Added advanced timing metrics for processors, such as CPU…

[markap14]

… Usage, time reading/writing content repo, process session commit time, etc. Exposed via nifi.sh diagnostics and made configurable via nifi.properties

Summary

NIFI-00000

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 8
  • JDK 11
  • JDK 17

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for introducing this new diagnostic capability @markap14! The implementation looks straightforward, just noted the opportunity for updating documentation, and aligning the internal default setting.

[exceptionfactory]

Thanks for making the adjustments @markap14! The tracking appears to work as designed, and provides some great information!

The only remaining concern is the length of each line in the tabular output. Perhaps using acronyms for column headers is one option, which would be very helpful when reading the diagnostics on smaller screens.

[exceptionfactory]

Understanding the need to balance readability with length, the current headers and column width produce very long lines in the diagnostic output, making it difficult to read without a very high resolution screen.

What do you think about using shorter names for most of the numeric fields, and then condensing the column width for those fields? Even the Processor Type field could be shortened, but the biggest gain would be for the numeric columns. Perhaps using acronyms that could be listed prior to the tabular output?

PS = Processing Seconds
CSP = CPU Seconds and percent time using CPU
PCTP = Percent CPU time used by Processor
DR = Disk Read Seconds
DW = Disk Write Seconds
SC = Session Commit Seconds
GC = Garbage Collection Milliseconds
MR = Megabytes Read
MW = Megabytes Written

[markap14]

-1. While I can appreciate the desire for conciseness, I think the readability is significantly hindered by reducing the column names to abbreviations. The reality is that if you're using NiFi, you generally are going to have a reasonably large screen, or the canvas is going to be too cramped anyway. And you can always grab the section out and throw it into a spreadsheet if you needed to. Additionally, the width needs to be fairly wide for some of these columns anyway. So having a heading like "MW" for a column that needs to be sufficiently wide to fit something like "4,820,182.54" is a bit awkward as well, when the heading could be explicit.

[exceptionfactory]

Thanks for the reply @markap14, that's a good point about the potential for larger numbers in column values. In light of the variety of options, and potential for different approaches down the road, leaving it as-is sounds good for now.

[exceptionfactory]

Thanks for addressing the feedback @markap14, the current version looks good! +1 merging

[markap14]

Thanks @exceptionfactory I appreciate you reviewing & working with me to get this merge.