Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make file state=absent respect recurse=false for previous state=directory #52656

Open
wants to merge 24 commits into
base: devel
Choose a base branch
from

Conversation

ewarnke
Copy link

@ewarnke ewarnke commented Feb 20, 2019

SUMMARY

Make file state=absent respect recuse=false for previos state=directory

Fixes #52653
Fixes #22248

ISSUE TYPE
  • Bugfix Pull Request

Sorry about the multiple pulls, not a frequent contributor.

@ewarnke
Copy link
Author

ewarnke commented Feb 20, 2019

Why is shippable using the wrong code branch!?

@ewarnke
Copy link
Author

ewarnke commented Feb 20, 2019

Tests need to be updated to set recurse=true when they want to delete non-empty directory

@ansibot

This comment has been minimized.

@ansibot ansibot added affects_2.8 This issue/PR affects Ansible v2.8 bug This issue/PR relates to a bug. merge_commit This PR contains at least one merge commit. Please resolve! module This issue/PR relates to a module. needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. needs_triage Needs a first human triage before being processed. new_contributor This PR is the first contribution by a new community member. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Feb 20, 2019
Copy link
Member

@dagwieers dagwieers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but this changes behaviour.

lib/ansible/modules/files/file.py Outdated Show resolved Hide resolved
lib/ansible/modules/files/file.py Outdated Show resolved Hide resolved
@dagwieers dagwieers changed the title Make file state=absent respect recuse=false for previos state=directory Make file state=absent respect recurse=false for previous state=directory Feb 20, 2019
@ansibot

This comment has been minimized.

@ansibot ansibot removed the needs_triage Needs a first human triage before being processed. label Feb 20, 2019
@ansibot

This comment has been minimized.

@ansibot ansibot removed merge_commit This PR contains at least one merge commit. Please resolve! needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html labels Feb 20, 2019
@ewarnke
Copy link
Author

ewarnke commented Feb 20, 2019

I get there are behavior changes but the current default behavior is unsafe. I'm open to suggestions on how to better do this without a complete rewrite of the core modules.

@jborean93 jborean93 added the ci_verified Changes made in this PR are causing tests to fail. label Feb 25, 2019
@jborean93
Copy link
Contributor

jborean93 commented Feb 25, 2019

CI test failure - https://app.shippable.com/github/ansible/ansible/runs/109917/63/tests

{
    "changed": false,
    "invocation": {
        "module_args": {
            "_diff_peek": null,
            "_original_basename": null,
            "access_time": null,
            "access_time_format": "%Y%m%d%H%M.%S",
            "attributes": null,
            "backup": null,
            "content": null,
            "delimiter": null,
            "directory_mode": null,
            "follow": true,
            "force": false,
            "group": null,
            "mode": null,
            "modification_time": null,
            "modification_time_format": "%Y%m%d%H%M.%S",
            "owner": null,
            "path": "/root/ansible_testing/foobar",
            "recurse": false,
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": null,
            "state": "absent",
            "unsafe_writes": null
        }
    },
    "msg": "rmtree failed: [Errno 39] Directory not empty: b'/root/ansible_testing/foobar'"
}

This is occurring for all the nodes running the file integration tests.

@dagwieers dagwieers dismissed their stale review February 25, 2019 10:58

Reported remarks have been fixed

@dagwieers
Copy link
Member

@ewarnke I agree, but the warning was intended for others reviewing this PR, not for you :-)

Copy link
Member

@dagwieers dagwieers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, however this PR changes the behaviour for existing users, on the other hand it fixes a security-related issue that could cause unintended data-loss. So IMO the impact of the latter trumps the former.

We may want to issue a warning and a deprecation period, rather than an immediate break in behaviour ?

@brontide
Copy link

Yeah, I've been thinking about abandoning these changes and working on a new pull. What's the thinking on new states, new argument, or new module?

  • States: absent_hard and absent_simple to introduce the new behaviors where absent_hard is the current behavior. That way documentation can be updated over time with the new behavior phased in. I don't like this because it makes a core module feel different.
  • Arguments: recursive_remove could be introduced to control the new behavior where it's yes by default for now and could be changed to no in a later release once people are given time to adjust. We could do a os.rmdir and if it fails, throw a warning about the changed behavior.
  • Module: Part of the problem is creation and destruction of files, links, and directories are subtly different. Maybe we should introduce a link and directory module that can better target these types of functions. That said the downside here is that you could not longer just pass a list of items to file state=absent and have them removed if they are files and directories.

I'm not tied to any one solution and the more I worked on this patch I'm just not happy with the way it's turning out. I'm fairly positive it would introduce major breakage as-is. I'm leaning towards a new argument right now since it seems like a good balance between fixing the bug and breaking existing behaviors while giving a path for people to select how directories are removed.

@brontide
Copy link

brontide commented Feb 25, 2019

@jborean93 Yeah, I've seen that and even tried chasing a few of the test down to add the flag. The problem is that the current behavior is unsafe and trying to find a good solution that merges cleanly and fixes the unsafe behavior without breaking piles of existing tasks is non-trivial. The fix is easy but people generally don't like breaking changes. - FYI, this is @ewarnke I just forgot to login under my work account.

@samdoran samdoran added the P3 Priority 3 - Approved, No Time Limitation label Feb 26, 2019
@bcoca bcoca removed the affects_2.8 This issue/PR affects Ansible v2.8 label Feb 26, 2019
@ansibot ansibot added the affects_2.9 This issue/PR affects Ansible v2.9 label Jun 1, 2019
@ansibot ansibot added the pre_azp This PR was last tested before migration to Azure Pipelines. label Dec 5, 2020
Copy link
Contributor

@s-hertel s-hertel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting this into the 2.12 cycle earlier rather than later is probably a good idea.

@@ -0,0 +1,7 @@
---
bugfixes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a breaking_changes: fragment since it's a very disruptive bugfix.

file:
path: '{{ output_dir }}'
state: absent
recurse: no
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A test deleting an empty directory with recurse: no would be nice.

- Recursively set the specified file attributes on directory contents.
- This applies only to C(state=directory).
- Recursively act on directory contents.
- This applies only to C(state=directory) or C(state=absent).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to note the version this started applying to C(state=absent), since it looks like the option has been around since 1.1.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This merge hasn't been updated in a year. It's virtually impossible to fix the unsafe behavior due to the fact that so many tests depend on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a lot of things depend on it. The core team discussed an open issue for it on Friday #22248 (comment) without being aware that this PR existed, and consensus was that we'd like to try to move forward with fixing it, as early as possible for the next release cycle so we can get feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ci_complete can be added to the latest commit to ensure all tests run.

@ansibot ansibot added stale_review Updates were made after the last review and the last review is more than 7 days old. has_issue labels Jul 12, 2023
@ansibot ansibot added needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html and removed needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html labels Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects_2.9 This issue/PR affects Ansible v2.9 bug This issue/PR relates to a bug. crypto Crypto community (ACME, openssl, letsencrypt) has_issue m:unarchive This issue/PR relates to the unarchive module. module This issue/PR relates to a module. needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. new_contributor This PR is the first contribution by a new community member. P3 Priority 3 - Approved, No Time Limitation packaging Packaging category pre_azp This PR was last tested before migration to Azure Pipelines. source_control Source-control category stale_review Updates were made after the last review and the last review is more than 7 days old. support:community This issue/PR relates to code supported by the Ansible community. support:core This issue/PR relates to code supported by the Ansible Engineering Team. test This PR relates to tests.
Projects
None yet