Skip to content

NIFI-9678 update Elasticsearch REST API processor integration-tests for Elasticsearch 8.x#5822

Closed
ChrisSamo632 wants to merge 1 commit into
apache:mainfrom
ChrisSamo632:NIFI-9678
Closed

NIFI-9678 update Elasticsearch REST API processor integration-tests for Elasticsearch 8.x#5822
ChrisSamo632 wants to merge 1 commit into
apache:mainfrom
ChrisSamo632:NIFI-9678

Conversation

@ChrisSamo632
Copy link
Copy Markdown
Contributor

Description of PR

NIFI-9678 test integration of Elasticsearch REST API processors with 8.x.

Tested using the following integration tests in the nifi-elasticsearch-client-service module (see the updated README.md):

# Elasticsearch 5
mvnd -P integration-tests,elasticsearch-oss clean verify

# Elasticsearch 6
sleep 2s
mvnd -P integration-tests,elasticsearch-oss,elasticsearch-6 clean verify

# Elasticsearch 7.10.2
sleep 2s
mvnd -P integration-tests,elasticsearch-oss,elasticsearch-7 clean verify
sleep 2s
mvnd -P integration-tests,elasticsearch-default,elasticsearch-7 clean verify

# Elasticsearch 7.17.1
sleep 2s
mvnd -P integration-tests,elasticsearch-default,elasticsearch-7-no-oss clean verify

# Elasticsearch 8.0.1
sleep 2s
mvnd -P integration-tests,elasticsearch-default,elasticsearch-8 clean verify

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

  • Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
  • Have you written or updated unit tests to verify your changes?
  • Have you verified that the full build is successful on JDK 8?
  • Have you verified that the full build is successful on JDK 11?
  • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
  • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
  • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

  • [ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

@gresockj gresockj self-requested a review March 2, 2022 12:54
String type = TYPE
if (VERSION.toString().startsWith("7.")) {
if (VERSION.toString().startsWith("8.")) {
// TODO: something that's deprecated when the 8.x branch progresses to include search-API deprecations
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What shall we do with this here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally not a fan of TODOs in code, but there's literally no deprecated Search API funcitons in Elasticsearch 8.x yet (it's too new for them to have deprecated anything, having just removed all previously deprecated stuff from 7.x).

I've Skipped the test for 8.x using the Assume but I think we should look to update this test to prove the WARNING funcitnoality for 8.x once Search API deprecations appear in Elasticsearch (so that would be a case of removing the Assume and implementing some sort of query here that caused a WARNING).

We could just remove this TODO and if branch, but I thought the TODO here would hopefully serve as a reminder to do this in future. I'd normally raise a Jira ticket to cover a TODO, but as there's no realistic way of currently implementing such a Jira ticket, it seemed like that wouldn't be sensible either.

So I think the options are:

  1. Leave as-is and pick this up for a future version of Elasticsearch 8.x
  2. Remove the TODO and hope someone picks this up in future
  3. Raise a Jira ticket to pick this up in future, noting that it can't currently be implemented and someone will need to watch out for an appropriate Search API deprecation in order to achieve the test

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the elaboration -- in light of that, I think we should keep the TODO.

Copy link
Copy Markdown
Contributor

@gresockj gresockj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I successfully ran the integration tests -- it's good to know ES 8 will be supported out of the box. Aside from one question about a leftover "TODO", this test update looks good to me.

Copy link
Copy Markdown
Contributor

@gresockj gresockj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification regarding the TODO, @ChrisSamo632 -- I'll go ahead and merge.

@asfgit asfgit closed this in 4cc20e6 Mar 2, 2022
krisztina-zsihovszki pushed a commit to krisztina-zsihovszki/nifi that referenced this pull request Jun 28, 2022
…or Elasticsearch 8.x

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes apache#5822.
Lehel44 pushed a commit to Lehel44/nifi that referenced this pull request Jul 1, 2022
…or Elasticsearch 8.x

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes apache#5822.
@ChrisSamo632 ChrisSamo632 deleted the NIFI-9678 branch October 21, 2022 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants