Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINIFICPP-1410 Add permissions property support for Putfile processor #942

Closed
wants to merge 10 commits into from

Conversation

lordgamez
Copy link
Contributor

There are multiple changes done in this PR:

  • Fixed Boost library usage, as previously even if libboost was present on the system it was not used. Now if EXCLUDE_BOOST is not defined and libboost is present on the system, it will be used marked with the USE_BOOST definition.
  • Tests with Boost usage moved from gcc-4.8 environment to ubuntu 20.04 as gcc-4.8 seems to be incompatible with Ubuntu 16.04 distribution's Boost library, causing segfaults and throwing std::bad_alloc exceptions.
  • Permissions and Directory Permissions properties added to Putfile processor for non-Windows systems for changing the permissions of the newly created output directories and the output file.
  • Default permissions for created directories are now 777 & umask instead of the previous 700 to be in sync with the boost implementation of boost::filesystem::create_directory.
  • Fixed permission problems using the new properties in docker tests.
  • Some additional filesystem functionalities are now supported by boost implementations.

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

Copy link
Contributor

@arpadboda arpadboda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, LGTM!


#ifndef WIN32
class FilePermissions {
static const uint32_t MINIMUM_INVALID_PERMISSIONS_VALUE = 1 << 9;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice solution :)

PROCESSORS.md Outdated
@@ -899,6 +899,8 @@ In the list below, the names of required properties appear in bold. Any other pr
|**Create Missing Directories**|true||If true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.|
|Directory|.||The output directory to which to put files<br/>**Supports Expression Language: true**|
|Maximum File Count|-1||Specifies the maximum number of files that can exist in the output directory|
|Permissions|||Sets the permissions on the output file to the value of this attribute. Format must be in octal number (e.g. 644 or 0755). Not supported on Windows systems.|
|Directory Permissions|||Sets the permissions on the directories being created if 'Create Missing Directories' property is set. Format must be format octal number (e.g. 644 or 0755). Not supported on Windows systems.|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: two "format"s. I would get rid of all "format"s and write "Must be an octal number (e.g. 644 or 0755)." in both property descriptions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 163bbea

core::Property PutFile::Permissions(
core::PropertyBuilder::createProperty("Permissions")
->withDescription("Sets the permissions on the output file to the value of this attribute. "
"Format must be format octal number (e.g. 644 or 0755). Not supported on Windows systems.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even more "format"s here :)
same suggestion as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 163bbea

core::PropertyBuilder::createProperty("Directory Permissions")
->withDescription("Sets the permissions on the directories being created if 'Create Missing Directories' property is set. "
"Format must be format octal number (e.g. 644 or 0755). Not supported on Windows systems.")
->build());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if these are not set? I would have expected a default value here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case it works the same way as boost::filesystem::create_directory which is creating the directory in mode 777 then applies the system's default umask.

return S_ISDIR(dir_stat.st_mode);
}

inline int exists(const std::string& path) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that returning 0 (true) and -1 (false) fits into the existing pattern, but I think it would be better to return a bool at least from is_directory() and exists().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 163bbea

@arpadboda arpadboda closed this in d027b62 Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants