MINIFICPP-1380 - Batch behavior for CompressContent and MergeContent processors #917
Conversation
| core::Property BinFiles::BatchSize( | ||
| core::PropertyBuilder::createProperty("Batch Size") | ||
| ->withDescription("Maximum number of FlowFiles processed in a single session") | ||
| ->withDefaultValue<uint32_t>(1)->build()); |
There was a problem hiding this comment.
Wonder if this is the best default value...
There was a problem hiding this comment.
I wanted to preserve the original behavior, as this is an optimazation feature the user can configure on an as-needed basis.
Should we change it, and if so, what do you think would be the appropriate default?
| auto flow = session->get(); | ||
|
|
||
| if (flow == nullptr) { | ||
| break; |
There was a problem hiding this comment.
Wonder if yield should be applied in case 0 FFs were processed
There was a problem hiding this comment.
the problem with yielding is that, this processor (MergeContent) can function correctly even if there are no incoming flowFiles (it can still emit already processed merged files)
this is not true for CompressContent so added a yield there
| #define FOR_EACH_7(fn, delim, _1, _2, _3, _4, _5, _6, _7) \ | ||
| fn(_1) delim() fn(_2) delim() fn(_3) delim() fn(_4) delim() \ | ||
| fn(_5) delim() fn(_6) delim() fn(_7) | ||
| #define FOR_EACH_8(fn, delim, _1, _2, _3, _4, _5, _6, _7, _8) \ |
There was a problem hiding this comment.
I wonder if it can be done in a generic way with variadic arg and using the first of the variadic part to process and pass the rest to a recursive call.
There was a problem hiding this comment.
I wish, but recursive macro expansion is a no-go 😢
There was a problem hiding this comment.
we could give something like this a try:
#define FOR_EACH_8(fn, delim, _1, ...) fn(_1) delim() FOR_EACH_7(fn, delim, __VA_ARGS__)
adamdebreceni
force-pushed
the
MINIFICPP-1380
branch
from
13e48b4
to
3a65799
Compare
adamdebreceni
force-pushed
the
MINIFICPP-1380
branch
from
3a65799
to
6392eb9
Compare
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.