Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve STREAMs speed, unify with SEND/RECEIVE #47

Open
fachat opened this issue Apr 5, 2020 · 3 comments
Open

Improve STREAMs speed, unify with SEND/RECEIVE #47

fachat opened this issue Apr 5, 2020 · 3 comments
Labels
api-breaking Changes that break the kernel API and require re-writing or at lease adapting some code enhancement New feature or request
Milestone

Comments

@fachat
Copy link
Owner

fachat commented Apr 5, 2020

The streams interface is horribly slow as it requires one kernel call per character for reading and another for writing. Previous attempts to speed up have not brought the necessary results (see the GETB/PUTB calls).

A new approach will be as follows:

  • STRMAP: map a user space buffer to the receiving resp. sending end of a stream. A buffer can be more than 256 byte in length.
  • STRPUSH / STRPULL to send resp. receive data into these mapped buffers, where only the number of bytes put to / taken from the buffer are exchanged as parameters, and the kernel copies the data between the user space and kernel space buffers

The buffer can be of different types:

  • contiguous: like a single fread() resp. fwrite() to/from a single buffer
  • circular: a circular buffer
  • send/receive: STRPUSH and STRPULL are interleaved and a two-way send/receive communication is established

Advantages:

  • (hopefully) better speed, e.g. lib6502 can map the full segment buffer and the only loop over STRPULLs on load. The filesystem can use a circular buffer, the kernel will translate.
  • the buffer mapping is stored in the kernel and does not need to be re-calculated every time
  • potential to extend to zero-copy transfers (but unclear yet how the remapping protocol will work)
  • the send/receive code can replace the global SYSBUF/PCBUF mechanism, and related semaphore handling

Disadvantages:

  • kernel/library size increases
  • only a single task should allow to map a buffer (for resource reasons). the others (if any) still need to use the single-byte interface. Devices will also use the single-byte interfaces
  • some sys calls /may/ require mapping the buffer on 256 byte boundaries (e.g. FORK) to avoid remapping in-between processing

Open issues:

  • send/receive protocol not fully clear yet
@fachat fachat added enhancement New feature or request api-breaking Changes that break the kernel API and require re-writing or at lease adapting some code labels Apr 5, 2020
@fachat
Copy link
Owner Author

fachat commented Apr 10, 2020

This would be API-breaking if SEND/RECEIVE are gone then

@fachat fachat added this to the 2.2 milestone May 2, 2020
@fachat
Copy link
Owner Author

fachat commented Sep 22, 2023

See the GeckOS-NG-Buffers.adoc design document https://github.com/fachat/GeckOS-V2/blob/master/GeckOS-NG-Buffers.adoc

@fachat
Copy link
Owner Author

fachat commented Sep 22, 2023

Note: plan to have this as a kernel addition in 2.2, while the actual breaking change will be in 3.0

@fachat fachat modified the milestones: 2.2, 2.3 Mar 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-breaking Changes that break the kernel API and require re-writing or at lease adapting some code enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant