Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-3594 Encrypted provenance repository implementation #1686

Closed
wants to merge 28 commits into from

Commits on Apr 20, 2017

  1. NIFI-3594 Added first unit test for PersistentProvenanceRepository op…

    …eration.
    
    Added BC dependency to nifi-persistent-provenance-repository module.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    d4de39b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8006d12 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f620003 View commit details
    Browse the repository at this point in the history
  4. NIFI-3594 Added skeleton of encrypted provenance repository (KeyProvi…

    …der w/ 2 impls, Encryptor skeleton, and exceptions/utilities).
    
    Reorganized tests to proper path.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    c497644 View commit details
    Browse the repository at this point in the history
  5. NIFI-3594 Added encryption methods and reflective property accessors.…

    … Pausing to re-evaluate because work may need to be done at lower level (EventWriter/EventReader -- byte/Object serialization).
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    2fdbb23 View commit details
    Browse the repository at this point in the history
  6. NIFI-3594 Intermediate changes before discussion with Mark Payne abou…

    …t intercepting SchemaRecordReader/Writer serialization (no updates to schema necessary).
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    5951b78 View commit details
    Browse the repository at this point in the history
  7. NIFI-3594 Moved (Keyed)CipherProvider classes & tests into nifi-secur…

    …ity-utils to include in nifi-data-provenance-utils.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    054cdef View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1dfbb5b View commit details
    Browse the repository at this point in the history
  9. NIFI-3594 Implemented encrypted read, write, and seek operations.

    Resolved RAT and checkstyle issues.
    All tests pass.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    2637a30 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    33db1ee View commit details
    Browse the repository at this point in the history
  11. NIFI-3594 Delgated reader and writer to use AESKeyedCipherProvider (e…

    …nhanced error checking and guard controls).
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    d844c00 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    7ae1a1c View commit details
    Browse the repository at this point in the history
  13. NIFI-3594 Working event encryptor lifecycle unit test with full encry…

    …ption metadata serialization.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    b912a14 View commit details
    Browse the repository at this point in the history
  14. NIFI-3594 Refactored AESProvenanceEventEncryptor implementation (remo…

    …ved cached ciphers to allow non-repeating IVs).
    
    Added unit tests.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    83523ad View commit details
    Browse the repository at this point in the history
  15. NIFI-3594 Added forAlgorithm static constructor for EncryptionMethod.

    Added validity checks for algorithm and version in AESProvenanceEventEncryptor.
    Added unit tests.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    60f4f03 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    b8314e8 View commit details
    Browse the repository at this point in the history
  17. NIFI-3594 Refactored key availability interface contract.

    Refactored encryptor composition.
    Added unit tests.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    09115b2 View commit details
    Browse the repository at this point in the history
  18. NIFI-3594 Began adding configuration properties for encrypted provena…

    …nce repository.
    
    Added utility methods for validation.
    Added unit tests.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    10b203e View commit details
    Browse the repository at this point in the history
  19. NIFI-3594 Added new NiFi properties keys for provenance repository en…

    …cryption.
    
    Added nifi.provenance.repository.encryption.key to default sensitive keys and updated unit tests and test resources.
    Added method to correctly calculate protected percentage of sensitive keys (unpopulated keys are no longer counted against protection %).
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    711bfa4 View commit details
    Browse the repository at this point in the history
  20. NIFI-3594 Implemented StaticKeyProvider and FileBasedKeyProvider.

    Moved getBestEventIdentifier() from StandardProvenanceEventRecord to ProvenanceEventRecord interface and added delegate in all implementations to avoid ClassCastException from multiple classloaders.
    Initialized IV before cipher to suppress unnecessary warnings.
    Added utility method to read encrypted provenance keys from key provider file.
    Suppressed logging of event record details in LuceneEventIndex.
    Added logic to create EncryptedSchemaRecordReader (if supported) in RecordReaders.
    Cleaned up EncryptedSchemaRecordReader and EncryptedSchemaRecordWriter.
    Added keyProvider, recordReaderFactory, and recordWriterFactory initialization to EncryptedWriteAheadProvenanceRepository to provide complete interceptor implementation.
    Added logic to RepositoryConfiguration to load encryption-related properties if necessary.
    Refactored WriteAheadProvenanceRepository to allow subclass implementation.
    Registered EncryptedWAPR in ProvenanceRepository implementations.
    Added unit tests for EWAPR.
    Added new nifi.properties keys for encrypted provenance repository.
    alopresto committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    092cb2d View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    578d0d1 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    b44b603 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2017

  1. NIFI-3594 Switched concatByteArrays implementation to manual concaten…

    …ation of arrays.
    
    Added unit test demonstrating performance improvement.
    alopresto committed Apr 24, 2017
    Configuration menu
    Copy the full SHA
    206e658 View commit details
    Browse the repository at this point in the history
  2. NIFI-3594 Improved byte[] handling code for performance/memory effici…

    …ency with Mark Payne's feedback.
    
    Cleaned up commented code.
    alopresto committed Apr 24, 2017
    Configuration menu
    Copy the full SHA
    fc7ed0c View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2017

  1. NIFI-3594 Added multiple key feature to StaticKeyProvider.

    Refactored StaticKeyProvider and FileBasedKeyProvider to reduce duplicate code.
    Added helper methods in NiFiProperties to read multiple key definitions for StaticKeyProvider.
    Fixed undetected NPE in tests (storing null value into properties).
    Added unit tests.
    alopresto committed Apr 25, 2017
    Configuration menu
    Copy the full SHA
    bbbabf8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1fc2ff2 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d1d79a1 View commit details
    Browse the repository at this point in the history
  4. NIFI-3594 Added Java JUnit test in nifi-data-provenance-utils module …

    …to trigger Maven running Groovy unit tests.
    alopresto committed Apr 25, 2017
    Configuration menu
    Copy the full SHA
    d6ff66c View commit details
    Browse the repository at this point in the history