Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: refactor rimraf retry options #30644

Merged
merged 5 commits into from Nov 27, 2019
Merged

fs: refactor rimraf retry options #30644

merged 5 commits into from Nov 27, 2019

Conversation

@cjihrig
Copy link
Contributor

cjihrig commented Nov 25, 2019

This PR implements the changes proposed in #30580.

This PR doesn't quite close out #30580 though because synchronous retries are not able to be fully implemented just yet (libuv/libuv#2548 should enable that soon though).

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
@cjihrig cjihrig force-pushed the cjihrig:rimraf branch from 61f3fb7 to bb07ba3 Nov 25, 2019
100ms longer on each try. This option represents the number of retries. This
option is ignored if the `recursive` option is not `true`. **Default:** `3`.
* `maxRetries` {integer} If an `EBUSY`, `EMFILE`, `ENOTEMPTY`, or `EPERM`
error is encountered, Node.js will retry the operation with a linear backoff

This comment has been minimized.

Copy link
@yorkie

yorkie Nov 25, 2019

Member

How about supporting exponential backoff or allow to custom, which may react to the real world.

This comment has been minimized.

Copy link
@cjihrig

cjihrig Nov 25, 2019

Author Contributor

I don't plan on adding that myself, and definitely not in this PR. But someone could open a follow up PR that passes in a user defined function for that.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@cjihrig

This comment has been minimized.

Copy link
Contributor Author

cjihrig commented Nov 27, 2019

ping for reviews

lib/internal/fs/rimraf.js Outdated Show resolved Hide resolved
@cjihrig cjihrig force-pushed the cjihrig:rimraf branch from bb07ba3 to 2476d13 Nov 27, 2019
@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@cjihrig

This comment has been minimized.

Copy link
Contributor Author

cjihrig commented Nov 27, 2019

The failures were test-http2-client-upload, which is a known flake, and test-inspector-wait-for-connection, which is #30619. I'm going to land this.

@cjihrig cjihrig mentioned this pull request Nov 27, 2019
2 of 4 tasks complete
cjihrig and others added 5 commits Nov 25, 2019
This is part of reworking the rimraf retry logic.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This commit makes retries an opt-in feature by defaulting
to no automatic retries. This will be particularly important
once synchronous operations can sleep between attempts.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This commit removes the emfileWait option. EMFILE errors are
now handled the same as any other retriable error.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This commit adds a retryDelay option to rimraf which configures
the amount of time between retry operations.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Co-authored-by: Thang Tran <trankimthang279@gmail.com>
Fixes: #30482
Refs: #30499
Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
@cjihrig cjihrig force-pushed the cjihrig:rimraf branch from 2476d13 to 74f8196 Nov 27, 2019
@cjihrig cjihrig merged commit 74f8196 into nodejs:master Nov 27, 2019
0 of 2 checks passed
0 of 2 checks passed
Travis CI - Branch Build Errored
Details
Travis CI - Pull Request Build Errored
Details
@cjihrig

This comment has been minimized.

Copy link
Contributor Author

cjihrig commented Nov 27, 2019

Landed in 84aa192...74f8196. Thanks for the reviews.

@cjihrig cjihrig deleted the cjihrig:rimraf branch Nov 27, 2019
addaleax added a commit that referenced this pull request Nov 30, 2019
This is part of reworking the rimraf retry logic.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax added a commit that referenced this pull request Nov 30, 2019
This commit makes retries an opt-in feature by defaulting
to no automatic retries. This will be particularly important
once synchronous operations can sleep between attempts.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax added a commit that referenced this pull request Nov 30, 2019
This commit removes the emfileWait option. EMFILE errors are
now handled the same as any other retriable error.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax added a commit that referenced this pull request Nov 30, 2019
This commit adds a retryDelay option to rimraf which configures
the amount of time between retry operations.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax added a commit that referenced this pull request Nov 30, 2019
Co-authored-by: Thang Tran <trankimthang279@gmail.com>
Fixes: #30482
Refs: #30499
Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
@targos

This comment has been minimized.

Copy link
Member

targos commented Dec 1, 2019

Should this be semver-major?

@cjihrig

This comment has been minimized.

Copy link
Contributor Author

cjihrig commented Dec 1, 2019

rimraf is still experimental, so it doesn't need to be.

bmeck added a commit to bmeck/node that referenced this pull request Dec 2, 2019
This is part of reworking the rimraf retry logic.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
bmeck added a commit to bmeck/node that referenced this pull request Dec 2, 2019
This commit makes retries an opt-in feature by defaulting
to no automatic retries. This will be particularly important
once synchronous operations can sleep between attempts.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
bmeck added a commit to bmeck/node that referenced this pull request Dec 2, 2019
This commit removes the emfileWait option. EMFILE errors are
now handled the same as any other retriable error.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
bmeck added a commit to bmeck/node that referenced this pull request Dec 2, 2019
This commit adds a retryDelay option to rimraf which configures
the amount of time between retry operations.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
bmeck added a commit to bmeck/node that referenced this pull request Dec 2, 2019
Co-authored-by: Thang Tran <trankimthang279@gmail.com>
Fixes: nodejs#30482
Refs: nodejs#30499
Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
@BridgeAR BridgeAR mentioned this pull request Dec 3, 2019
BridgeAR added a commit that referenced this pull request Dec 3, 2019
Notable changes:

* fs:
  * Reworked experimental recursive `rmdir()`  (cjihrig)
    #30644
    * The `maxBusyTries` option is renamed to `maxRetries`, and its
      default is set to 0. The `emfileWait` option has been removed,
      and `EMFILE` errors use the same retry logic as other errors.
      The `retryDelay` option is now supported. `ENFILE` errors are
      now retried.
* http:
  * Make maximum header size configurable per-stream or per-server
    (Anna Henningsen) #30570
* http2:
  * Make maximum tolerated rejected streams configurable (Denys
    Otrishko) #30534
  * Allow to configure maximum tolerated invalid frames (Denys
    Otrishko) #30534
* wasi:
  * Introduce initial WASI support (cjihrig)
    #30258

PR-URL: #30774
BridgeAR added a commit that referenced this pull request Dec 3, 2019
Notable changes:

* fs:
  * Reworked experimental recursive `rmdir()`  (cjihrig)
    #30644
    * The `maxBusyTries` option is renamed to `maxRetries`, and its
      default is set to 0. The `emfileWait` option has been removed,
      and `EMFILE` errors use the same retry logic as other errors.
      The `retryDelay` option is now supported. `ENFILE` errors are
      now retried.
* http:
  * Make maximum header size configurable per-stream or per-server
    (Anna Henningsen) #30570
* http2:
  * Make maximum tolerated rejected streams configurable (Denys
    Otrishko) #30534
  * Allow to configure maximum tolerated invalid frames (Denys
    Otrishko) #30534
* wasi:
  * Introduce initial WASI support (cjihrig)
    #30258

PR-URL: #30774
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
This is part of reworking the rimraf retry logic.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
This commit makes retries an opt-in feature by defaulting
to no automatic retries. This will be particularly important
once synchronous operations can sleep between attempts.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
This commit removes the emfileWait option. EMFILE errors are
now handled the same as any other retriable error.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
This commit adds a retryDelay option to rimraf which configures
the amount of time between retry operations.

Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
Co-authored-by: Thang Tran <trankimthang279@gmail.com>
Fixes: nodejs#30482
Refs: nodejs#30499
Refs: nodejs#30580
PR-URL: nodejs#30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sebastien-Ahkrin added a commit to Sebastien-Ahkrin/node that referenced this pull request Dec 11, 2019
Notable changes:

* fs:
  * Reworked experimental recursive `rmdir()`  (cjihrig)
    nodejs#30644
    * The `maxBusyTries` option is renamed to `maxRetries`, and its
      default is set to 0. The `emfileWait` option has been removed,
      and `EMFILE` errors use the same retry logic as other errors.
      The `retryDelay` option is now supported. `ENFILE` errors are
      now retried.
* http:
  * Make maximum header size configurable per-stream or per-server
    (Anna Henningsen) nodejs#30570
* http2:
  * Make maximum tolerated rejected streams configurable (Denys
    Otrishko) nodejs#30534
  * Allow to configure maximum tolerated invalid frames (Denys
    Otrishko) nodejs#30534
* wasi:
  * Introduce initial WASI support (cjihrig)
    nodejs#30258

PR-URL: nodejs#30774
targos added a commit that referenced this pull request Jan 13, 2020
This is part of reworking the rimraf retry logic.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos added a commit that referenced this pull request Jan 13, 2020
This commit makes retries an opt-in feature by defaulting
to no automatic retries. This will be particularly important
once synchronous operations can sleep between attempts.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos added a commit that referenced this pull request Jan 13, 2020
This commit removes the emfileWait option. EMFILE errors are
now handled the same as any other retriable error.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos added a commit that referenced this pull request Jan 13, 2020
This commit adds a retryDelay option to rimraf which configures
the amount of time between retry operations.

Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos added a commit that referenced this pull request Jan 13, 2020
Co-authored-by: Thang Tran <trankimthang279@gmail.com>
Fixes: #30482
Refs: #30499
Refs: #30580
PR-URL: #30644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.