New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmapstorage #2106

Closed
mpranj opened this Issue Jun 20, 2018 · 5 comments

Comments

Projects
None yet
2 participants
@mpranj
Contributor

mpranj commented Jun 20, 2018

Meta issue for tracking mmapstorage. (a.k.a. notes to myself)

Implementation:

  • refactor & cleanup after writing more tests
  • replace MAP_ANONYMOUS with malloc when unlinking

Testing:

  • move generic storage plugin tests out of mmap and test all storage plugins
  • check code coverage

Builds:

  • jenkins build job with mmapstorage as default
  • travis-ci build job with mmapstorage as default
@mpranj

This comment has been minimized.

Contributor

mpranj commented Aug 6, 2018

@markus2330 would it be OK to use the XSI extension for mmapstorage? I would like to use uintptr_t for some of my pointer trickery. It seems to be the only integral type that is guaranteed to be able to hold the integer value of a pointer (and can be cast back and forth). I used size_t but noticed that there are no guarantees.

Which architectures should mmap support? Only 32/64bit, or more? If so, which ones. My byte-swap detection / endianness check depends a bit on that.

@markus2330

This comment has been minimized.

Contributor

markus2330 commented Aug 6, 2018

According to http://pubs.opengroup.org/onlinepubs/007904875/basedefs/xbd_chap02.html#tag_02_01_04 also mmap is part of XSI. So it should be fine to use uintptr_t.

About architectures: It is important that we detect any differences in architectures but it is not important that the mmap files of a different architecture can be used. The goal of the mmap plugins is to be a local cache. It is okay that you discard mmap files that stem from other architectures. If it is fast and easy to do some byte swapping, it is of course okay to do so. But you should concentrate on the core features before doing such extra features.

@mpranj

This comment has been minimized.

Contributor

mpranj commented Aug 6, 2018

I don’t want to do byte swaps. The magic number to detect different endianness with arbitrary byte swaps would be to give each byte a different value. (Thus being able to detect an arbitrary byte swap)

If we only support limited architectures (2-4), it is probably simpler to hardcode the magic numbers instead of generating them dynamically in the plugin open function or similar.

@markus2330

This comment has been minimized.

Contributor

markus2330 commented Aug 6, 2018

We should detect any byte swaps. So better we generate a bit more data and we can be sure that nobody will experience nasty bugs.

@mpranj

This comment has been minimized.

Contributor

mpranj commented Sep 9, 2018

Resolved via #1990.

@mpranj mpranj closed this Sep 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment