NIFI-12231 FetchSmb supports Move and Delete Completion Strategies

[turcsanyip]

Summary

NIFI-12231

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

The integration tests workflow failed on FetchSmbIT for the following tests:

FetchSmbIT.testCompletionStrategyDelete:114 expected: <false> but was: <true>
FetchSmbIT.testCompletionStrategyMoveWithCreatingDirectory:149 expected: <false> but was: <true>

[turcsanyip]

The integration tests workflow failed on FetchSmbIT for the following tests:

FetchSmbIT.testCompletionStrategyDelete:114 expected: <false> but was: <true>
FetchSmbIT.testCompletionStrategyMoveWithCreatingDirectory:149 expected: <false> but was: <true>

Thanks for the heads-up @exceptionfactory!
I rerun the integration test on GitHub and got the same errors. So they are not intermittent.
However, I did not manage to replicate it in my environments. It worked on Mac and now I also set up a similar environment to GitHub Actions (same OS, Java and Docker versions) but no luck.
I pushed some changes in order to get more info and also modified the logic a bit (just trial-and-error approach).

[turcsanyip]

@exceptionfactory I managed to fix the tests by changing the logic in FetchSmb.

There were 2 overlapping sessions to the SMB server: one for reading the file and the other for executing the completion strategy (move or delete) in commitAsync(). I moved the commit out of the TWR block of the first session, so the "read" session gets closed before the "move/delete" session starts.

Parallel sessions should not cause issues as the file handles are closed properly and it worked in my environments (ran flows connecting to Windows share and Samba share, ran integrations tests on Mac and Linux). Not sure what the docker environment is doing in Github CI but this change fixed it. I re-ran the integration tests 3 times and all were green.