Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utils: drop boost::interprocess::file_lock #13862

Merged
merged 1 commit into from Aug 29, 2018

Conversation

Projects
None yet
6 participants
@ken2812221
Copy link
Member

commented Aug 3, 2018

boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.

This PR is seperated from #13426 for easier review.

@DrahtBot

This comment has been minimized.

Copy link
Contributor

commented Aug 3, 2018

Note to reviewers: This pull request conflicts with the following ones:
  • #13878 (utils: Add fstream wrapper to allow to pass unicode filename on Windows by ken2812221)
  • #13866 (utils: Use _wfopen and _wfreopen on Windows by ken2812221)
  • #13746 (-masterdatadir for datadir bootstrapping by kallewoof)
  • #13743 (refactor: Replace std/boost::bind with lambda by ken2812221)
  • #13159 (Don't close old debug log file handle prematurely when trying to re-open (on SIGHUP) by practicalswift)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@Empact

This comment has been minimized.

Copy link
Member

commented Aug 3, 2018

Seems like a good opportunity to drop boost::interprocess::file_lock altogether, no?

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch 2 times, most recently Aug 3, 2018

src/fs.h Outdated
class FileLock
{
public:
FileLock(const fs::path& file);

This comment has been minimized.

Copy link
@Empact

Empact Aug 3, 2018

Member

nit: could delete default and copy constructor

This comment has been minimized.

Copy link
@ken2812221

ken2812221 Aug 3, 2018

Author Member

Fixed

src/fs.cpp Outdated
lock.l_start = 0;
lock.l_len = 0;
return -1 != fcntl(fd, F_SETLK, &lock);
}

This comment has been minimized.

Copy link
@Empact

Empact Aug 3, 2018

Member

The relevant boost implementation, for reference:

inline bool try_acquire_file_lock(file_handle_t hnd, bool &acquired)
{
   struct ::flock lock;
   lock.l_type    = F_WRLCK;
   lock.l_whence  = SEEK_SET;
   lock.l_start   = 0;
   lock.l_len     = 0;
   int ret = ::fcntl(hnd, F_SETLK, &lock);
   if(ret == -1){
      return (errno == EAGAIN || errno == EACCES) ?
               acquired = false, true : false;
   }
   return (acquired = true);
}

https://www.boost.org/doc/libs/1_67_0/boost/interprocess/detail/os_file_functions.hpp

Should we distinguish between hard (e.g. EINVAL) and soft (EAGAIN, EACCES) failures, as they do? We could throw in the former case?
https://www.gnu.org/software/libc/manual/html_node/File-Locks.html

src/fs.cpp Outdated
}
_OVERLAPPED overlapped = {0};
return LockFileEx((HANDLE)hFile, LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY, 0, 0, 0, &overlapped);
}

This comment has been minimized.

Copy link
@Empact

Empact Aug 3, 2018

Member

The relevant boost implementation, for reference:

inline bool try_acquire_file_lock(file_handle_t hnd, bool &acquired)
{
   const unsigned long len = ((unsigned long)-1);
   winapi::interprocess_overlapped overlapped;
   std::memset(&overlapped, 0, sizeof(overlapped));
   if(!winapi::lock_file_ex
      (hnd, winapi::lockfile_exclusive_lock | winapi::lockfile_fail_immediately,
       0, len, len, &overlapped)){
      return winapi::get_last_error() == winapi::error_lock_violation ?
               acquired = false, true : false;

   }
   return (acquired = true);
}

https://www.boost.org/doc/libs/1_67_0/boost/interprocess/detail/os_file_functions.hpp

Similar question re. GetLastError?

This comment has been minimized.

Copy link
@Empact

Empact Aug 3, 2018

Member

Also is the (HANDLE) cast necessary?

This comment has been minimized.

Copy link
@ken2812221

ken2812221 Aug 3, 2018

Author Member

Fixed

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch Aug 3, 2018

@ken2812221

This comment has been minimized.

Copy link
Member Author

commented Aug 3, 2018

@Empact In my opinion, we deal catching exception and try_lock fail much the same when we use boost, so no need to handle the extra exception for now.

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch Aug 3, 2018

@ken2812221 ken2812221 changed the title add unicode compatible file_lock for Windows utils: drop boost::interprocess::file_lock Aug 3, 2018

@Empact

This comment has been minimized.

Copy link
Member

commented Aug 3, 2018

@ken2812221 FYI boost does throw on the failures noted above:

inline bool file_lock::try_lock()
{
   bool result;
   if(!ipcdetail::try_acquire_file_lock(m_file_hnd, result)){
      error_info err(system_error_code());
      throw interprocess_exception(err);
   }
   return result;
}

namespace boost {
namespace interprocess {
inline int system_error_code() // artifact of POSIX and WINDOWS error reporting
{
   #if (defined BOOST_INTERPROCESS_WINDOWS)
   return winapi::get_last_error();
   #else
   return errno; // GCC 3.1 won't accept ::errno
   #endif
}

https://www.boost.org/doc/libs/1_67_0/boost/interprocess/sync/file_lock.hpp
https://www.boost.org/doc/libs/1_46_1/boost/interprocess/errors.hpp

@ken2812221

This comment has been minimized.

Copy link
Member Author

commented Aug 3, 2018

@Empact I simply return false in my code at those conditions.

@Empact

This comment has been minimized.

Copy link
Member

commented Aug 3, 2018

Yeah so the net effect is the loss of this message:
return error("Error while attempting to lock directory %s: %s", directory.string(), e.what());

Certainly not a big deal but breadcrumbs can mean the difference between an easy fix and a maddening mystery.

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch Aug 3, 2018

@ken2812221

This comment has been minimized.

Copy link
Member Author

commented Aug 3, 2018

@Empact Updated to get error messages.

src/fs.cpp Outdated
return false;
}
_OVERLAPPED overlapped = {0};
return LockFileEx(hFile, LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY, 0, 0, 0, &overlapped);

This comment has been minimized.

Copy link
@donaloconnor

donaloconnor Aug 3, 2018

Contributor

The CreateFileW with 0 sharing effectively opens the file as exclusive (locking it). It feels the TryLock() here is redundant because we don't require locking of specific bytes.

Perhaps the TryLock() should be the one doing the CreateFileW.
Apologies if I've misunderstood something here :-)

This comment has been minimized.

Copy link
@ken2812221

ken2812221 Aug 3, 2018

Author Member

Fixed

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch Aug 3, 2018

src/fs.cpp Outdated
@@ -12,4 +21,82 @@ FILE *freopen(const fs::path& p, const char *mode, FILE *stream)
return ::freopen(p.string().c_str(), mode, stream);
}

#ifndef WIN32

std::string GetErrorReason() {

This comment has been minimized.

Copy link
@Empact

Empact Aug 4, 2018

Member

nit: static

src/fs.cpp Outdated
}
#else

std::string GetErrorReason() {

This comment has been minimized.

Copy link
@Empact

Empact Aug 4, 2018

Member

nit: static

return error("Error while attempting to lock directory %s: %s", directory.string(), e.what());
auto lock = MakeUnique<fsbridge::FileLock>(pathLockFile);
if (!lock->TryLock()) {
return error("Error while attempting to lock directory %s: %s", directory.string(), lock->GetReason());

This comment has been minimized.

Copy link
@Empact

Empact Aug 4, 2018

Member

For reference, how these are logged by the callers:

bitcoin/src/init.cpp

Lines 1183 to 1185 in 2b67354

if (!LockDirectory(datadir, ".lock", probeOnly)) {
return InitError(strprintf(_("Cannot obtain a lock on data directory %s. %s is probably already running."), datadir.string(), _(PACKAGE_NAME)));
}

bitcoin/src/wallet/db.cpp

Lines 136 to 139 in 2b67354

if (!LockDirectory(pathIn, ".walletlock")) {
LogPrintf("Cannot obtain a lock on wallet directory %s. Another instance of bitcoin may be using it.\n", strPath);
return false;
}

@ken2812221

This comment has been minimized.

Copy link
Member Author

commented Aug 5, 2018

Update: use std codecvt instead of custom converter.

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch 9 times, most recently Aug 18, 2018

@laanwj

This comment has been minimized.

Copy link
Member

commented Aug 27, 2018

lightly tested ACK 80119487840db0c3944a89f92772660ee594c4c3

tested on-

  • Linux Ubuntu 18.04 ()
  • OpenBSD 6.3 ()
  • FreeBSD 11.2 ( )
src/fs.cpp Outdated
lock.l_whence = SEEK_SET;
lock.l_start = 0;
lock.l_len = 0;
if (-1 == fcntl(fd, F_SETLK, &lock)) {

This comment has been minimized.

Copy link
@laanwj

laanwj Aug 27, 2018

Member

would prefer simply fcntl(fd, F_SETLK, &lock) == -1 (more consistent with the other if statements in the code)

This comment has been minimized.

Copy link
@ken2812221

ken2812221 Aug 27, 2018

Author Member

fixed

add unicode compatible file_lock for Windows
boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.
This commit add a new class to handle those specific file for Windows.

@ken2812221 ken2812221 force-pushed the ken2812221:custom-filelock branch to 1661a47 Aug 27, 2018

@laanwj laanwj merged commit 1661a47 into bitcoin:master Aug 29, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Aug 29, 2018

Merge #13862: utils: drop boost::interprocess::file_lock
1661a47 add unicode compatible file_lock for Windows (Chun Kuan Lee)

Pull request description:

  boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.

  This PR is seperated from #13426 for easier review.

Tree-SHA512: e240479cda65958bf6e1319840b83928b2b50da81d99f4f002fb3b62621370bcd4bcfacd2b8c0678c443a650d6ba53d9d12618b591e5bfd67ac14388a18fd822

@ken2812221 ken2812221 deleted the ken2812221:custom-filelock branch Aug 29, 2018

sipa added a commit that referenced this pull request Oct 20, 2018

Merge #14426: utils: Fix broken Windows filelock
369244f utils: Fix broken Windows filelock (Chun Kuan Lee)

Pull request description:

  Fix broken filelock on Windows, also add a test for this. It's a regression introduced by #13862.

Tree-SHA512: 15665b1930cf39ec71f3ab07def8e2897659f6fd4d2de749d63a5a8ec920e4a04282f12bc262f242b1b3d14d2dd9fa191ddbcf16a46fb927b5b2b14d9f6b5d01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.