Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-5346 EncryptPGP, DecryptPGP, SignPGP, and VerifyPGP Processors #4032

Closed
wants to merge 4 commits into from

Conversation

natural
Copy link
Contributor

@natural natural commented Feb 3, 2020

EncryptPGP, DecryptPGP, SignPGP, and VerifyPGP Processors and PGPKeyMaterialControllerService

Enables PGP encryption, decryption, digital signing and verification functionality; fixes bug NIFI-5346.

Also addresses or resolves NIFI-5335, NIFI-6708.

The code in this change set introduces 4 new PGP processors and a related PGP key material service.

The processors and service classes are no doubt in the wrong place; help naming them + putting them in better places is very much appreciated.

The change set also includes a processor benchmark and reporting class , but that isn't wired together at the moment.

There are unit tests to cover the basic use cases, and of course there will be more added.

Basic docs are included, more pending feedback.

Benchmarks for EncryptContent and the new EncryptPGP and DecryptPGP processors:

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptContent/PBE   PGP+armor               398 ms       2634 b/ms
2   EncryptContent/PBE   PGP+armor               227 ms       4619 b/ms
3   EncryptContent/PBE   PGP+armor               381 ms       2752 b/ms
4   EncryptContent/PBE   PGP+armor               308 ms       3404 b/ms
5   EncryptContent/PBE   PGP+armor               305 ms       3437 b/ms
-----------------------------------------------------------------------
fastest: 227 ms @ 4,619 b/ms | slowest: 398 ms | mean: 323 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptContent/PBE   PGP                     191 ms       5489 b/ms
2   EncryptContent/PBE   PGP                     363 ms       2888 b/ms
3   EncryptContent/PBE   PGP                     104 ms      10082 b/ms
4   EncryptContent/PBE   PGP                     220 ms       4766 b/ms
5   EncryptContent/PBE   PGP                     274 ms       3826 b/ms
-----------------------------------------------------------------------
fastest: 104 ms @ 10,082 b/ms | slowest: 363 ms | mean: 230 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE CAMELLIA 256            361 ms       2904 b/ms
2   EncryptPGP+DecryptPGP/PBE CAMELLIA 256            505 ms       2076 b/ms
3   EncryptPGP+DecryptPGP/PBE CAMELLIA 256            319 ms       3287 b/ms
4   EncryptPGP+DecryptPGP/PBE CAMELLIA 256            245 ms       4279 b/ms
5   EncryptPGP+DecryptPGP/PBE CAMELLIA 256            499 ms       2101 b/ms
-----------------------------------------------------------------------
fastest: 245 ms @ 4,279 b/ms | slowest: 505 ms | mean: 385 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE AES 128                 256 ms       4096 b/ms
2   EncryptPGP+DecryptPGP/PBE AES 128                 338 ms       3102 b/ms
3   EncryptPGP+DecryptPGP/PBE AES 128                 355 ms       2953 b/ms
4   EncryptPGP+DecryptPGP/PBE AES 128                 241 ms       4350 b/ms
5   EncryptPGP+DecryptPGP/PBE AES 128                 191 ms       5489 b/ms
-----------------------------------------------------------------------
fastest: 191 ms @ 5,489 b/ms | slowest: 355 ms | mean: 276 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE DES                     486 ms       2157 b/ms
2   EncryptPGP+DecryptPGP/PBE DES                     153 ms       6853 b/ms
3   EncryptPGP+DecryptPGP/PBE DES                     479 ms       2189 b/ms
4   EncryptPGP+DecryptPGP/PBE DES                     507 ms       2068 b/ms
5   EncryptPGP+DecryptPGP/PBE DES                     165 ms       6355 b/ms
-----------------------------------------------------------------------
fastest: 153 ms @ 6,853 b/ms | slowest: 507 ms | mean: 358 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE CAMELLIA 128            481 ms       2179 b/ms
2   EncryptPGP+DecryptPGP/PBE CAMELLIA 128            160 ms       6553 b/ms
3   EncryptPGP+DecryptPGP/PBE CAMELLIA 128            165 ms       6355 b/ms
4   EncryptPGP+DecryptPGP/PBE CAMELLIA 128            146 ms       7182 b/ms
5   EncryptPGP+DecryptPGP/PBE CAMELLIA 128            193 ms       5433 b/ms
-----------------------------------------------------------------------
fastest: 146 ms @ 7,182 b/ms | slowest: 481 ms | mean: 229 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE IDEA                    528 ms       1985 b/ms
2   EncryptPGP+DecryptPGP/PBE IDEA                    165 ms       6355 b/ms
3   EncryptPGP+DecryptPGP/PBE IDEA                    360 ms       2912 b/ms
4   EncryptPGP+DecryptPGP/PBE IDEA                    336 ms       3120 b/ms
5   EncryptPGP+DecryptPGP/PBE IDEA                    487 ms       2153 b/ms
-----------------------------------------------------------------------
fastest: 165 ms @ 6,355 b/ms | slowest: 528 ms | mean: 375 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE AES 256                 341 ms       3075 b/ms
2   EncryptPGP+DecryptPGP/PBE AES 256                 163 ms       6432 b/ms
3   EncryptPGP+DecryptPGP/PBE AES 256                 382 ms       2744 b/ms
4   EncryptPGP+DecryptPGP/PBE AES 256                 391 ms       2681 b/ms
5   EncryptPGP+DecryptPGP/PBE AES 256                 453 ms       2314 b/ms
-----------------------------------------------------------------------
fastest: 163 ms @ 6,432 b/ms | slowest: 453 ms | mean: 346 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE CAMELLIA 192            211 ms       4969 b/ms
2   EncryptPGP+DecryptPGP/PBE CAMELLIA 192            203 ms       5165 b/ms
3   EncryptPGP+DecryptPGP/PBE CAMELLIA 192            453 ms       2314 b/ms
4   EncryptPGP+DecryptPGP/PBE CAMELLIA 192            289 ms       3628 b/ms
5   EncryptPGP+DecryptPGP/PBE CAMELLIA 192            374 ms       2803 b/ms
-----------------------------------------------------------------------
fastest: 203 ms @ 5,165 b/ms | slowest: 453 ms | mean: 306 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE CAST5                   182 ms       5761 b/ms
2   EncryptPGP+DecryptPGP/PBE CAST5                   268 ms       3912 b/ms
3   EncryptPGP+DecryptPGP/PBE CAST5                   441 ms       2377 b/ms
4   EncryptPGP+DecryptPGP/PBE CAST5                   281 ms       3731 b/ms
5   EncryptPGP+DecryptPGP/PBE CAST5                   509 ms       2060 b/ms
-----------------------------------------------------------------------
fastest: 182 ms @ 5,761 b/ms | slowest: 509 ms | mean: 336 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE TRIPLE DES              450 ms       2330 b/ms
2   EncryptPGP+DecryptPGP/PBE TRIPLE DES              155 ms       6765 b/ms
3   EncryptPGP+DecryptPGP/PBE TRIPLE DES              208 ms       5041 b/ms
4   EncryptPGP+DecryptPGP/PBE TRIPLE DES              247 ms       4245 b/ms
5   EncryptPGP+DecryptPGP/PBE TRIPLE DES              441 ms       2377 b/ms
-----------------------------------------------------------------------
fastest: 155 ms @ 6,765 b/ms | slowest: 450 ms | mean: 300 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE BLOWFISH                312 ms       3360 b/ms
2   EncryptPGP+DecryptPGP/PBE BLOWFISH                203 ms       5165 b/ms
3   EncryptPGP+DecryptPGP/PBE BLOWFISH                280 ms       3744 b/ms
4   EncryptPGP+DecryptPGP/PBE BLOWFISH                204 ms       5140 b/ms
5   EncryptPGP+DecryptPGP/PBE BLOWFISH                250 ms       4194 b/ms
-----------------------------------------------------------------------
fastest: 203 ms @ 5,165 b/ms | slowest: 312 ms | mean: 249 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE AES 192                 459 ms       2284 b/ms
2   EncryptPGP+DecryptPGP/PBE AES 192                 206 ms       5090 b/ms
3   EncryptPGP+DecryptPGP/PBE AES 192                 390 ms       2688 b/ms
4   EncryptPGP+DecryptPGP/PBE AES 192                 252 ms       4161 b/ms
5   EncryptPGP+DecryptPGP/PBE AES 192                 268 ms       3912 b/ms
-----------------------------------------------------------------------
fastest: 206 ms @ 5,090 b/ms | slowest: 459 ms | mean: 315 ms

#   Test                 Parameter             Duration           Rate
=======================================================================
1   EncryptPGP+DecryptPGP/PBE TWOFISH                 509 ms       2060 b/ms
2   EncryptPGP+DecryptPGP/PBE TWOFISH                 234 ms       4481 b/ms
3   EncryptPGP+DecryptPGP/PBE TWOFISH                 194 ms       5405 b/ms
4   EncryptPGP+DecryptPGP/PBE TWOFISH                 304 ms       3449 b/ms
5   EncryptPGP+DecryptPGP/PBE TWOFISH                 382 ms       2744 b/ms
-----------------------------------------------------------------------
fastest: 194 ms @ 5,405 b/ms | slowest: 509 ms | mean: 324 ms

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically master)?

  • Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

  • Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
  • Have you written or updated unit tests to verify your changes?
  • Have you verified that the full build is successful on both JDK 8 and JDK 11?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
  • If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
  • If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@alopresto
Copy link
Contributor

Reviewing...

@alopresto
Copy link
Contributor

I'm getting a lot of compilation errors on Java 8. I see some language constructs that weren't introduced until Java 9, like Set.of(). I started fixing them but there are more than I anticipated. Can you please provide a new commit which also builds on Java 8?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants