NIFI-13146 ConsumeSlack rate limit error mitigation

[krisztina-zsihovszki]

Summary

NIFI-13146

ConsumeSlack processor got stuck in initialization phase and reported rate limit error continuously when it was running in a Slack workplace with thousands of channels.

As a mitigation for the problem I added a logic which checks if "Channels" property contains only IDs. In case no channel name is specified, another Slack API call (conversations.info) is used to fetch the channel names for the channel IDs and no need to fetch the whole conversation list.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • [x ] JDK 21

UI Contributions

• NiFi is modernizing its UI. Any contributions that update the current UI also need to be implemented in the new UI.

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[dan-s1]

I am not sure the possible number of channels but it does not seem right to have so many log statements on info level here, line 285 and line 294.

[krisztina-zsihovszki]

@dan-s1 Thank you for the comment.
The info logging on line 285 and line 294 were added to the original implementation, I just moved the block, on line 268 I tried to follow the original logging method in the new code.

I think there won't be so many channels consumed at the same time, but I am open to change the level of logging to debug.

[krisztina-zsihovszki]

@markap14 If you have free cycles, can you please check this PR? You created the original implementation, do you think this modification makes sense? Thanks.

[markap14]

Thanks @krisztina-zsihovszki the changes seems reasonable. Contrib-check looks good, and I was able to verify behavior using both ID's and channel names. +1 will merge.