Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fields with a state of DROP on normalization failure #2267

Commits on Feb 7, 2024

  1. Fields with a state of DROP on normalization failure are not written …

    …to the index but are still preserved in the event.
    rdhayes68 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    6f55333 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. Create docker images in CI (#2264)

    alerman authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    2911976 View commit details
    Browse the repository at this point in the history
  2. updated common-utils submodule and updated quickstart docker to not c…

    …lean up old images
    jwomeara authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    0fd8475 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    93ddde2 View commit details
    Browse the repository at this point in the history
  4. Enforce term limits during EdgeQueries (#2274)

    * Enforce term limits during EdgeQueries
    * Update error message
    * Fix EdgeQuery maxQueryTerms setting
    jschmidt10 authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    4c51cb6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3e80d99 View commit details
    Browse the repository at this point in the history
  6. Added remote edge query logic and tests (#2250)

    * Added remote edge query logic and tests
    * factored the BaseRemoteQueryLogic out of the remote edge and event instances
    ivakegg authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    262cdde View commit details
    Browse the repository at this point in the history
  7. Updates to try and avoid hanging ivarators (#2093)

    * Updates to try and avoid hanging ivarators
    * fixed configuration issues of the thread pools
    * Updates to allow value to be set via properties.
    ivakegg authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    35561c1 View commit details
    Browse the repository at this point in the history
  8. Clearing noisy logs in start-loader (#2258)

    * Cleaning noisy logs in start-loader
    
    * Removing tracer log calls
    
    * Adjusting logs for additional loader
    
    * Small adjustment to echo
    
    ---------
    
    Co-authored-by: pcagbu <pcagbu@ip-10-113-15-132.evoforge.org>
    2 people authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    7afbc59 View commit details
    Browse the repository at this point in the history
  9. 1799 add codestyle (#1811)

    - added git pre-push hook to ensure code is formatted properly
    
    Co-authored-by: Christopher Gross cogross <christopher.gross@red-drum.com>
    2 people authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    8a05595 View commit details
    Browse the repository at this point in the history
  10. Use the pluginEnvironment instead of AccumuloConfiguration as the (#2200

    )
    
    * Use the pluginEnvironment instead of AccumuloConfiguration as the
    later class will break when tables are dropped
    * Updated tests to handle use of PluginEnvironment
    * Reapply TablePropKey error fix to ConfigurableAgeOffFilter
    ivakegg authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    13f196e View commit details
    Browse the repository at this point in the history
  11. Enrich 'i' and 'ri' rows in metadata table with event date (#2094)

    * Enrich 'i' and 'ri' rows in metadata table with event date
    
    Modify the generation of 'i' (indexed rows) and 'ri' (reverse indexed
    rows) in the metadata table such that the column qualifier contains the
    event date. This is required as a first step to support efforts for
    issue #825 so that we can identify dates when an event was ingested and
    included in a frequency count for an associated 'f' row, but was not
    indexed.
    
    * Add counts to 'i' and 'ri' rows
    lbschanno authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    cf9d8cf View commit details
    Browse the repository at this point in the history
  12. 6.7.0-SNAPSHOT

    ivakegg authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    efb4e28 View commit details
    Browse the repository at this point in the history
  13. #2226 - fix javadoc warnings (#2232)

    * fix javadoc warnings
    
    ---------
    
    Co-authored-by: Christopher Gross cogross <christopher.gross@red-drum.com>
    2 people authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    05a4a12 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    4a92fa8 View commit details
    Browse the repository at this point in the history
  15. 2170 - update dependencies (#2176)

    * 2170 - update dependencies
    ---------
    Co-authored-by: Christopher Gross cogross <christopher.gross@red-drum.com>
    cogross authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    6fbd222 View commit details
    Browse the repository at this point in the history
  16. 2013 - jexl uuid query parser research/changes

    Christopher Gross cogross authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    2f370d3 View commit details
    Browse the repository at this point in the history
  17. Improved SSDeep ingest and query test framework (#2230)

    * Replaced custom Tuple2 implementation with ImmutablePair from commons-lang3
    * Includes complete ingest testframework for SSDeep data that includes both shard data and ssdeepIndex data.
    * Cleans up some sins of the past related to the faceted query logic leaking into the test framework.
    * Adds protoype datawave ingest SSDeepIndexHandler, good enough for testing at this point.
    * Trimmed pom for ssdeep-ingest to minimum dependency set
    * Reorganized package structure to keep ssdeep code/tests together
    * Moved SSDeepQueryTest to common package
    * Fix parent pom version for ingest-ssdeep and static version for ssdeep-common
    * Addressed comments from code review
    drewfarris authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    2a269b3 View commit details
    Browse the repository at this point in the history
  18. Update LookupUUIDUtil.java

    alerman authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    cd3f1e1 View commit details
    Browse the repository at this point in the history
  19. Fix pruning logic for IngestTypePruningVisitor after removal of JexlN…

    …odes.children() method
    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    464b2db View commit details
    Browse the repository at this point in the history
  20. Re-introduce RebuildingScannerTestHelper to QueryTestTableHelper (#2279)

    * @jwomeara observed that the `QueryTestTableHelper` was no longer using the `RebuildingScannerTestHelper`, and was using a regular `InMemoryInstance` instead, so we were not testing teardowns and interrupts.
    * Updated `RebuildingScannerTestHelper` to use `AccumuloClient` instead of `AccumuloConnection` incorporating changes from the `feature/queryMicroservices` branch
    * The only test broken by this is the `SSDeepIngestQueryTest`, so `@Ignore`-ing for now until that test can be fixed by me.
    
    Co-authored-by: Whitney O'Meara <jwomeara@users.noreply.github.com>
    2 people authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    c046a45 View commit details
    Browse the repository at this point in the history
  21. Updated metadata-utils submodule

    jwomeara authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    6be206a View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    81d1cb3 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    13ef78b View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    fad9276 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    66aa834 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    c6ab9c7 View commit details
    Browse the repository at this point in the history
  27. Add missed module

    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    06b9fe2 View commit details
    Browse the repository at this point in the history
  28. Update comments

    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    0d46590 View commit details
    Browse the repository at this point in the history
  29. Update unit tests to junit5

    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    e356252 View commit details
    Browse the repository at this point in the history
  30. Add junit-bom depedency and use version of junit-bom for explicit jup…

    …iter and vintage dependencies
    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    a29c463 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    ffb87b3 View commit details
    Browse the repository at this point in the history
  32. Additional module cleanup

    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    a801876 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    ea86d85 View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    2360406 View commit details
    Browse the repository at this point in the history
  35. add junit-jupiter-engine to common-test

    add junit-jupiter-engine to warehouse-common
    
    remove jupiter-api from warehouse pom
    
    restore jupitier-api to ws-client pom
    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    7d497f1 View commit details
    Browse the repository at this point in the history
  36. fix bad rebase

    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    684cab9 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    2c62646 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    5b9c1bf View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    e77f54c View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    36956c6 View commit details
    Browse the repository at this point in the history
  41. remove duplicate junit-jupiter-engine dependency from warehouse-commo…

    …n pom, add jupiter-engine and vintage-engine to failsafe plugins in webservices and warehouse pom
    apmoriarty authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    7e356d1 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    97c7b60 View commit details
    Browse the repository at this point in the history
  43. re #2270: Iimplementation of a record filter for the ingest processing (

    #2271)
    
    * Created an event predicate for discarding events during ingest processing
    * Created a discard interval and future discard interval predicates
    * Updated tests to test hit and miss on the interval predicates
    * Add default for DISCARD_FUTURE_INTERVAL event
    
    Co-authored-by: foster33 <lukefos62@gmail.com>
    Co-authored-by: Keith Ratcliffe <ratcliffe.keith@gmail.com>
    Co-authored-by: hlgp <hlgp@users.noreply.github.com>
    Co-authored-by: palindrome <31748527+hlgp@users.noreply.github.com>
    Co-authored-by: matthpeterson <matt.peterson@tibertechnologies.com>
    6 people authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    6470036 View commit details
    Browse the repository at this point in the history
  44. Update pom's for 6.8.0-SNAPSHOT

    hgklohr authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    dcb6a28 View commit details
    Browse the repository at this point in the history
  45. Make the configcheck module optional as it is a separate utility not (#…

    …2283)
    
    required in the main datawave builds
    ivakegg authored and rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    cbbfa38 View commit details
    Browse the repository at this point in the history
  46. Fields with a state of DROP on normalization failure are not written …

    …to the index but are still preserved in the event.
    rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    81a66c8 View commit details
    Browse the repository at this point in the history
  47. Merge branch 'integration' into 2266-drop-state-for-normalization-fai…

    …lure-incorrectly-removes-the-field-from-the-event-as-well
    rdhayes68 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    7ae8d9c View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. Update code format

    rdhayes68 committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    abf29bd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    caa0632 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'origin/2266-drop-state-for-normalizatio…

    …n-failure-incorrectly-removes-the-field-from-the-event-as-well' into 2266-drop-state-for-normalization-failure-incorrectly-removes-the-field-from-the-event-as-well
    rdhayes68 committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    8499d97 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    41ef8f3 View commit details
    Browse the repository at this point in the history