Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle the result of posix_fallocate system call #15650

Merged
merged 1 commit into from May 2, 2019
Merged

Handle the result of posix_fallocate system call #15650

merged 1 commit into from May 2, 2019
+4 −3

Conversation

lucayepa
Copy link
Contributor

The system call posix_fallocate is not supported on some filesystems.

  • catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

Fixes #15624

@gmaxwell
Copy link
Contributor

OR-ing the fallback version with linux might be clearer than sticking in a bunch of returns and leaving in a chunk of dead code that some static analysis tools may warn about.

@lucayepa
Copy link
Contributor Author

Do you think we can consider the Linux part in the fallback? I don't know if posix_allocate can be called on every platform. It will become something like:

#if defined(WIN32)
...
#elif defined(MAC_OSX)
...
#endif
    // Linux and fallback version
 (this calls posix_fallocate for every other platform)

This way the system call posix_fallocate would be called for every fallback platform. I don't know if this is a good idea.

@DrahtBot
Copy link
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #14485 (Try to use posix_fadvise with CBufferedFile by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@gmaxwell
Copy link
Contributor

No but you can do
#if win32
#elif mac
#else
#if linux
#endif
fallback
#endif

@lucayepa
Copy link
Contributor Author

This sounds great. I indent the nested #if to be consistent with the style of the rest of the code.

I also add here a link for reference to the first introduction of the system call: #2229.

@gmaxwell
Copy link
Contributor

Style ACK. :) (should get some testing on an impacted filesystem)

@maflcko
Copy link
Member

maflcko commented Mar 23, 2019

utACK eb18d9a

@maflcko maflcko added this to the 0.19.0 milestone Mar 23, 2019
@lucayepa
Copy link
Contributor Author

lucayepa commented Mar 23, 2019 •
edited

Just tested on an affected filesystem (unionf on Debian stable), using strace to check the system calls sent to the system. The success return value of posix_fallocate is obviously zero. Somehow in a previous version I had the '!', but then I pushed the wrong one, without it. I've just force-pushed the good version, I've just tested with strace.

Affected filesystem (fallocate result is EOPNOTSUPP, and the fallback code is activated):

$ strace test/test_bitcoin -t flatfile_tests 2>&1 | grep -A 5 fallocate | tail -6
fallocate(3, 0, 0, 100)                 = -1 EOPNOTSUPP (Operation not supported)
fcntl(3, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
fstatfs(3, {f_type=FUSE_SUPER_MAGIC, f_bsize=4096, f_blocks=2047598, f_bfree=712591, f_bavail=603651, f_files=524288, f_ffree=411273, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_NOSUID|ST_NODEV|ST_RELATIME}) = 0
pwrite64(3, "\0", 1, 99)                = 1
close(3)                                = 0

Ext4 filesystem (fallocate result is zero, that means success, and no other code is activated):

$ strace test/test_bitcoin -t flatfile_tests 2>&1 | grep -A 1 fallocate | tail -2
fallocate(3, 0, 0, 100)                 = 0
close(3)                                = 0

maflcko
maflcko reviewed Mar 23, 2019
View reviewed changes
src/util/system.cpp Outdated Show resolved Hide resolved
@DrahtBot
Copy link
Contributor

Gitian builds for commit 7b13c64 (master):

Gitian builds for commit a48eb1b (master and this pull):

@maflcko
Copy link
Member

maflcko commented Mar 25, 2019

utACK 5d35ae3

@sipa
Copy link
Member

sipa commented Mar 25, 2019

utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.

luke-jr
luke-jr approved these changes Apr 18, 2019
View reviewed changes
Copy link
Member

@luke-jr luke-jr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably would be better to have configure check for posix_fallocate availability instead of using __linux__, but since it's already that way, this is strictly an improvement.

utACK

@hebasto
Copy link
Member

hebasto commented May 2, 2019

utACK 5d35ae3

1 similar comment
@practicalswift
Copy link
Contributor

utACK 5d35ae3

@maflcko maflcko merged commit 5d35ae3 into bitcoin:master May 2, 2019
maflcko pushed a commit that referenced this pull request May 2, 2019
Merge #15650: Handle the result of posix_fallocate system call
c4560a7 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes #15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request May 2, 2019
@sidhujag
Merge bitcoin#15650: Handle the result of posix_fallocate system call
b5fe08a 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
@hebasto hebasto mentioned this pull request May 2, 2019
Closed
luke-jr pushed a commit to luke-jr/bitcoin that referenced this pull request Aug 23, 2019
@lucayepa @luke-jr
Handle the result of posix_fallocate system call
1fd11c5 
Github-Pull: bitcoin#15650
Rebased-From: 5d35ae3
@luke-jr luke-jr mentioned this pull request Aug 23, 2019
Merged
fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Aug 24, 2019
@lucayepa @fanquake
Handle the result of posix_fallocate system call
39a0452 
Github-Pull: bitcoin#15650
Rebased-From: 5d35ae3
fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Sep 23, 2019
@lucayepa @fanquake
Handle the result of posix_fallocate system call
c52dd12 
Github-Pull: bitcoin#15650
Rebased-From: 5d35ae3
laanwj added a commit that referenced this pull request Nov 25, 2019
@laanwj
Merge #16617: [0.18.2] Backports
29d7026 
0b18ea6 util: Filter control characters out of log messages (Wladimir J. van der Laan)
ac30fc4 build: Factor out qt translations from build system (Wladimir J. van der Laan)
3b8af5f build: update boost macros to latest upstream (fanquake)
b12defc Test that joinpsbts randomly shuffles the inputs (Andrew Chow)
eb07d22 Shuffle inputs and outputs after joining psbts (Andrew Chow)
1175410 addrdb: Remove temporary files created in SerializeFileDB. Fixes non-determinism in unit tests. (practicalswift)
c52dd12 Handle the result of posix_fallocate system call (Luca Venturini)
f792b25 torcontrol: Use the default/standard network port for Tor hidden services, even if the internal port is set differently (Luke Dashjr)
9fe8d28 Bugfix: QA: Run tests with UPnP disabled (Luke Dashjr)
1d12e52 Add vertical spacer (Josu Goñi)
d764141 depends: add patch to common dependencies (fanquake)
56815e9 Give QApplication dummy arguments (Andrew Chow)
9d389d0 util: No translation of `Bitcoin Core` in the copyright (MarcoFalke)
87908e9 scripted-diff: Avoid passing PACKAGE_NAME for translation (MarcoFalke)
a44e18f build: Stop translating PACKAGE_NAME (MarcoFalke)
7bd8f4e rpc: Fix getblocktemplate CLI example (#16594) (Emil Engler)
1cc06a1 doc: Fix typos in COPYRIGHT (Chuf)

Pull request description:

  Backports some commits to the `0.18` branch:

  * #16596 - rpc: Fix getblocktemplate CLI example
  * #16615 - doc: Fix typos in COPYRIGHT
  * #16291 - gui: Stop translating PACKAGE_NAME (without the `make translate` commit)
  * #16578 - Do not pass in command line arguments to QApplication
  * #16051 - depends: add patch to common dependencies
  * #16090 - Add vertical spacer
  * #15651 - torcontrol: Use the default/standard network port for Tor hidden services, even if the internal port is set differently
  * #15650 - Handle the result of posix_fallocate system call
  * #16646 - Bugfix: QA: Run tests with UPnP disabled
  * #16212 - addrdb: Remove temporary files created in SerializeFileDB. Fixes non-determinism in unit tests.
  * #16512 - rpc: Shuffle inputs and outputs after joining psbts
  * #16870 - build: update boost macros to latest upstream for improved error reporting
  * #16982 - build: Factor out qt translations from build system
  * #17095 - util: Filter control characters out of log messages

ACKs for top commit:
  laanwj:
    ACK 0b18ea6

Tree-SHA512: 37f0e5afc20975f4d1506e8662eda2ae0125f2f424a852818b5af2c3b8db78fc1c365b83571aa80ca63c885ca314302190b891a50ff3851fda9b9238455a5627
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jun 27, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
b504998 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jun 28, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
7d35d71 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jun 29, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
a3d5e83 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 1, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
c29a551 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 1, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
02534d4 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 8, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
43a027d 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 10, 2021
@PastaPastaPasta
Merge bitcoin#15650: Handle the result of posix_fallocate system call
0199129 
5d35ae3 Handle the result of posix_fallocate system call (Luca Venturini)

Pull request description:

  The system call `posix_fallocate` is not supported on some filesystems.

  - catches the result of posix_allocate and fall back to the default behaviour if the return value is different from 0 (success)

  Fixes bitcoin#15624

ACKs for commit 5d35ae:
  MarcoFalke:
    utACK 5d35ae3
  sipa:
    utACK 5d35ae3, though the Yoda condition is an uncommon style in this project.
  hebasto:
    utACK 5d35ae3
  practicalswift:
    utACK 5d35ae3

Tree-SHA512: 7ab3b35fb633926f28a58b2b07ffde8e31bb997c80a716b1b45ee716fe9ff4ddcef0a05810bd4423530e220cfc62f8925517d27a8b92b05a524272063e43f746
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@maflcko maflcko maflcko left review comments

@luke-jr luke-jr luke-jr approved these changes

Assignees
No one assigned
Labels
Utils/log/libs
Projects
None yet
Milestone
0.19.0
Development

Successfully merging this pull request may close these issues.

flatfile_tests fail on alpine linux
9 participants
@lucayepa @gmaxwell @DrahtBot @maflcko @sipa @hebasto @practicalswift @luke-jr @fanquake