Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

index: release the merge lock if a merge raises an exception #312

Merged
merged 3 commits into from
Apr 28, 2021

Conversation

craigfe
Copy link
Member

@craigfe craigfe commented Apr 28, 2021

This has no effect on passing tests, but causes failing ones to behave more sensibly in the short term. A more complete solution would fix #284 entirely.

src/index.ml Outdated Show resolved Hide resolved
@craigfe craigfe force-pushed the test-hook-release-semaphore-lock branch 3 times, most recently from 3e73ade to 3fd6edd Compare April 28, 2021 13:49
@craigfe craigfe changed the title index: release the merge lock if a test hook raises an exception index: release the merge lock if a merge raises an exception Apr 28, 2021
Copy link
Member

@samoht samoht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - there are probably more code simplification that can be done, but that's not critical

src/index.ml Outdated Show resolved Hide resolved
@craigfe craigfe force-pushed the test-hook-release-semaphore-lock branch from 3fd6edd to 1bd7437 Compare April 28, 2021 13:57
@craigfe craigfe force-pushed the test-hook-release-semaphore-lock branch from 5581c86 to c1a7cf5 Compare April 28, 2021 14:07
@craigfe
Copy link
Member Author

craigfe commented Apr 28, 2021

Rebased.

match t.index with
| None -> Tbl.length log.mem
| Some index ->
(Int63.to_int (IO.offset index.io) / Entry.encoded_size)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is broken in 32 bit but we probably do not care :-)

@samoht
Copy link
Member

samoht commented Apr 28, 2021

LGTM

craigfe added a commit to craigfe/index that referenced this pull request Apr 28, 2021
Fixes the bug spotted here:
mirage#312 (comment). I think
`Int63.to_int` is a little too tempting from an API perspective, so I've
shadowed it and provided more clearly named checked and unchecked
alternatives.
craigfe added a commit to craigfe/index that referenced this pull request Apr 28, 2021
Fixes the bug spotted here:
mirage#312 (comment). I think
`Int63.to_int` is a little too tempting from an API perspective, so I've
shadowed it and provided more clearly named checked and unchecked
alternatives.
@samoht samoht merged commit d7c6329 into mirage:master Apr 28, 2021
samoht added a commit to samoht/opam-repository that referenced this pull request Apr 29, 2021
CHANGES:

## Fixed

- Reduce allocations during merge (mirage/index#274, mirage/index#277)

- Protect concurrent syncs with a lock (mirage/index#309)

- Fixed a performance issue for `Index.sync` when there is a blocking merge in
  progress: the `log_async` file was not cached properly and fully reloaded
  from disk every time. (mirage/index#310)

- Release the merge lock if a merge raises an exception (mirage/index#312)

- Added fsync after `Index.clear` to signal more quickly to read-only instances
  than something has changed in the file (mirage/index#308)

## Changed

- Specialise `IO.v` to create read-only or read-write instances. (mirage/index#291)

- `clear` removes the files on disks and opens new ones containing only the
  header. (mirage/index#288, mirage/index#307, mirage/index#317)
samoht added a commit to samoht/opam-repository that referenced this pull request Apr 29, 2021
CHANGES:

## Fixed

- Reduce allocations during merge (mirage/index#274, mirage/index#277)

- Protect concurrent syncs with a lock (mirage/index#309)

- Fixed a performance issue for `Index.sync` when there is a blocking merge in
  progress: the `log_async` file was not cached properly and fully reloaded
  from disk every time. (mirage/index#310)

- Release the merge lock if a merge raises an exception (mirage/index#312)

- Added fsync after `Index.clear` to signal more quickly to read-only instances
  than something has changed in the file (mirage/index#308)

## Changed

- Specialise `IO.v` to create read-only or read-write instances. (mirage/index#291)

- `clear` removes the files on disks and opens new ones containing only the
  header. (mirage/index#288, mirage/index#307, mirage/index#317)
samoht added a commit to samoht/opam-repository that referenced this pull request Apr 29, 2021
CHANGES:

## Fixed

- Reduce allocations during merge (mirage/index#274, mirage/index#277)

- Protect concurrent syncs with a lock (mirage/index#309)

- Fixed a performance issue for `Index.sync` when there is a blocking merge in
  progress: the `log_async` file was not cached properly and fully reloaded
  from disk every time. (mirage/index#310)

- Release the merge lock if a merge raises an exception (mirage/index#312)

- Added fsync after `Index.clear` to signal more quickly to read-only instances
  than something has changed in the file (mirage/index#308)

## Changed

- Specialise `IO.v` to create read-only or read-write instances. (mirage/index#291)

- `clear` removes the files on disks and opens new ones containing only the
  header. (mirage/index#288, mirage/index#307, mirage/index#317)
samoht added a commit to samoht/opam-repository that referenced this pull request Apr 29, 2021
CHANGES:

- Reduce allocations during merge (mirage/index#274, mirage/index#277)

- Protect concurrent syncs with a lock (mirage/index#309)

- Fixed a performance issue for `Index.sync` when there is a blocking merge in
  progress: the `log_async` file was not cached properly and fully reloaded
  from disk every time. (mirage/index#310)

- Release the merge lock if a merge raises an exception (mirage/index#312)

- Added fsync after `Index.clear` to signal more quickly to read-only instances
  than something has changed in the file (mirage/index#308)

- Specialise `IO.v` to create read-only or read-write instances. (mirage/index#291)

- `clear` removes the files on disks and opens new ones containing only the
  header. (mirage/index#288, mirage/index#307, mirage/index#317)
craigfe added a commit to craigfe/index that referenced this pull request Apr 29, 2021
Fixes the bug spotted here:
mirage#312 (comment). I think
`Int63.to_int` is a little too tempting from an API perspective, so I've
shadowed it and provided more clearly named checked and unchecked
alternatives.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure that asynchronous failures propagate cleanly
2 participants