Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINIFICPP-1008 - Chunkio integration into nanofi #761

Closed
wants to merge 7 commits into from

Conversation

msharee9
Copy link
Contributor

@msharee9 msharee9 commented Apr 16, 2020

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically master)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Broken down MINIFICPP-1007 into ECU core and C2 separately. This PR has ECU core changes with chunkio and libyaml integration and deleted the prior in memory buffer implementation.
Some more improvements has to be made:

  1. Optimized usage of chunkio chunks.
  2. Configurable flush interval and coordinate output flush tasks with input. (This will improve overall cpu and memory usage performance) and will scale to number of ecus.

…tegration

and libyaml c library for parsing ecu configuration file.
This work is in an effort to integrate chunkio for nanofi and to breakdown
the larger MINIFICPP-1007 and separate ECU core and C2 work into multiple
PRs.

MINIFICPP-1008 Adding ecu config yaml

MINIFICPP-1008 Added ecu yaml configuration parsing

MINIFICPP-1008 build on windows

MINIFICPP-1008 some changes to cleanup

MINIFICPP-1008 Replaced in memory buffer with chunkio buffering, handle backlog chunks at startup

MINIFICPP-1008 Handle backlogged chunks

MINIFICPP-1008 Build on windows

MINIFICPP-1008 Resolved some bugs
@msharee9 msharee9 changed the title Minificpp 1008 - Chunkio integration into nanofi MINIFICPP-1008 - Chunkio integration into nanofi Apr 16, 2020
@msharee9 msharee9 changed the title MINIFICPP-1008 - Chunkio integration into nanofi WIP: MINIFICPP-1008 - Chunkio integration into nanofi Apr 19, 2020
@msharee9 msharee9 marked this pull request as ready for review April 19, 2020 00:18
@msharee9 msharee9 changed the title WIP: MINIFICPP-1008 - Chunkio integration into nanofi MINIFICPP-1008 - Chunkio integration into nanofi Apr 20, 2020
@martinzink
Copy link
Member

I'm closing this due to inactivity and nanofi is already kinda deprecated.

@martinzink martinzink closed this Apr 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants