Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Eio.Buf_write #235

Merged
merged 3 commits into from
Jun 27, 2022
Merged

Add Eio.Buf_write #235

merged 3 commits into from
Jun 27, 2022

Conversation

talex5
Copy link
Collaborator

@talex5 talex5 commented Jun 25, 2022 •
edited
Loading

This is a buffered writer. It's based on Faraday, with a few changes:

  • with_flow wraps an Eio sink, flushing to it automatically when the writing fiber blocks.
  • Bigstrings have been replaced with Cstructs in the API, for consistency with the rest of Eio.
  • I replaced yield with pause and unpause, as it was unclear to me what it should do.
  • flush now waits for the flush to complete instead of taking a callback (since we now have effects).
  • operation is now await_batch. It now waits until data is available to write instead of returning Yield.
  • shift_buffers and shift_flushes are now tail-recursive.
  • Fixed overflow bug in shift_flushes.
  • write_string and write_bytes now check the range is valid.
  • serialize no longer hangs if it gets receives Closed.

@talex5 talex5 force-pushed the buf_write branch 2 times, most recently from 2619def to 956cddd Compare June 26, 2022 10:37
@talex5 talex5 force-pushed the buf_write branch 8 times, most recently from 19fd102 to cb23a45 Compare June 27, 2022 12:53
@talex5
Add Buf_write.with_flow
73ec23c 
`with_flow` wraps an Eio sink, flushing to it automatically when the writing fiber blocks.
This is intended to be then normal way of using `Buf_write` from Eio.

Some other changes to Faraday:

- I replaced `yield` with `pause` and `unpause`, as it was unclear to me what it should do.
- `flush` now waits for the flush to complete instead of taking a callback (since we now have effects).
- `operation` is now `await_batch`. It now waits until data is available to write instead of returning `Yield`.
- `shift_buffers` and `shift_flushes` are now tail-recursive.
- Fixed overflow bug in `shift_flushes`.
- Removed `write_` prefix from write functions, as it's now in the module name.
- `string` and `bytes` now check the range is valid.
- `serialize` no longer hangs if it gets receives `Closed.
@talex5 talex5 merged commit 03cbe41 into ocaml-multicore:main Jun 27, 2022
@talex5 talex5 deleted the buf_write branch June 27, 2022 13:14
@talex5 talex5 mentioned this pull request Jun 27, 2022
Open
talex5 added a commit to talex5/opam-repository that referenced this pull request Jun 28, 2022
@talex5
[new release] eio_main, eio_luv, eio_linux and eio (0.3)
cb5e170 
CHANGES:

API changes:

- `Net.accept_sub` is deprecated in favour of `accept_fork` (@talex5 ocaml-multicore/eio#240).
  `Fiber.fork_on_accept`, which it used internally, has been removed.

- Allow short writes in `Read_source_buffer` (@talex5 ocaml-multicore/eio#239).
  The reader is no longer required to consume all the data in one go.
  Also, add `Linux_eio.Low_level.writev_single` to expose this behaviour directly.

- `Eio.Unix_perm` is now `Eio.Dir.Unix_perm`.

New features:

- Add `Eio.Mutex` (@TheLortex @talex5 ocaml-multicore/eio#223).

- Add `Eio.Buf_write` (@talex5 ocaml-multicore/eio#235).
  This is a buffered writer for Eio sinks, based on Faraday.

- Add `Eio_mock` library for testing (@talex5 ocaml-multicore/eio#228).
  At the moment it has mock flows and networks.

- Add `Eio_mock.Backend` (@talex5 ocaml-multicore/eio#237 ocaml-multicore/eio#238).
  Allows running tests without needing a dependency on eio_main.
  Also, as it is single-threaded, it can detect deadlocks in test code instead of just hanging.

- Add `Buf_read.{of_buffer, of_string, parse_string{,_exn}, return}` (@talex5 ocaml-multicore/eio#225).

- Add `<*>` combinator to `Buf_read.Syntax` (@talex5 ocaml-multicore/eio#227).

- Add `Eio.Dir.read_dir` (@patricoferris @talex5 ocaml-multicore/eio#207 ocaml-multicore/eio#218 ocaml-multicore/eio#219)

Performance:

- Add `Buf_read` benchmark and optimise it a bit (@talex5 ocaml-multicore/eio#230).

- Inline `Buf_read.consume` to improve performance (@talex5 ocaml-multicore/eio#232).

Bug fixes / minor changes:

- Allow IO to happen even if a fiber keeps yielding (@TheLortex @talex5 ocaml-multicore/eio#213).

- Fallback for `traceln` without an effect handler (@talex5 ocaml-multicore/eio#226).
  `traceln` now works outside of an event loop too.

- Check for cancellation when creating a non-protected child context (@talex5 ocaml-multicore/eio#222).

- eio_linux: handle EINTR when calling `getrandom` (@bikallem ocaml-multicore/eio#212).

- Update to cmdliner.1.1.0 (@talex5 ocaml-multicore/eio#190).
talex5 added a commit to talex5/opam-repository that referenced this pull request Jun 28, 2022
@talex5
[new release] eio_main, eio_luv, eio_linux and eio (0.3)
5ad4967 
CHANGES:

API changes:

- `Net.accept_sub` is deprecated in favour of `accept_fork` (@talex5 ocaml-multicore/eio#240).
  `Fiber.fork_on_accept`, which it used internally, has been removed.

- Allow short writes in `Read_source_buffer` (@talex5 ocaml-multicore/eio#239).
  The reader is no longer required to consume all the data in one go.
  Also, add `Linux_eio.Low_level.writev_single` to expose this behaviour directly.

- `Eio.Unix_perm` is now `Eio.Dir.Unix_perm`.

New features:

- Add `Eio.Mutex` (@TheLortex @talex5 ocaml-multicore/eio#223).

- Add `Eio.Buf_write` (@talex5 ocaml-multicore/eio#235).
  This is a buffered writer for Eio sinks, based on Faraday.

- Add `Eio_mock` library for testing (@talex5 ocaml-multicore/eio#228).
  At the moment it has mock flows and networks.

- Add `Eio_mock.Backend` (@talex5 ocaml-multicore/eio#237 ocaml-multicore/eio#238).
  Allows running tests without needing a dependency on eio_main.
  Also, as it is single-threaded, it can detect deadlocks in test code instead of just hanging.

- Add `Buf_read.{of_buffer, of_string, parse_string{,_exn}, return}` (@talex5 ocaml-multicore/eio#225).

- Add `<*>` combinator to `Buf_read.Syntax` (@talex5 ocaml-multicore/eio#227).

- Add `Eio.Dir.read_dir` (@patricoferris @talex5 ocaml-multicore/eio#207 ocaml-multicore/eio#218 ocaml-multicore/eio#219)

Performance:

- Add `Buf_read` benchmark and optimise it a bit (@talex5 ocaml-multicore/eio#230).

- Inline `Buf_read.consume` to improve performance (@talex5 ocaml-multicore/eio#232).

Bug fixes / minor changes:

- Allow IO to happen even if a fiber keeps yielding (@TheLortex @talex5 ocaml-multicore/eio#213).

- Fallback for `traceln` without an effect handler (@talex5 ocaml-multicore/eio#226).
  `traceln` now works outside of an event loop too.

- Check for cancellation when creating a non-protected child context (@talex5 ocaml-multicore/eio#222).

- eio_linux: handle EINTR when calling `getrandom` (@bikallem ocaml-multicore/eio#212).

- Update to cmdliner.1.1.0 (@talex5 ocaml-multicore/eio#190).
@talex5 talex5 mentioned this pull request Jun 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@talex5