Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need check for large file support #1032

Closed
tp-m opened this issue Nov 12, 2016 · 9 comments
Closed

Need check for large file support #1032

tp-m opened this issue Nov 12, 2016 · 9 comments

Comments

@tp-m
Copy link
Member

tp-m commented Nov 12, 2016

I think we should have some kind of replacement for AC_SYS_LARGEFILE. This is something very common, but there may be more things in future.

So question is how/where to best do this.

  1. Just always check for it and add the required defines/flags to global/project args, and enable by default.

  2. Provide a method in meson main

  3. Add a new sys module or such and add a method there

  4. Anything else?

@nirbheek
Copy link
Member

My opinion is that large file support is always available these days (was introduced in 1996), so we should just enable it without checking for it.

The other question is whether there are legitimate uses where people don't want to enable large file support even when it is available. I don't think there are, but it's possible that there are such uses. We can either add support for disabling it from the start, or just always enable it and change it back if people complain.

@nirbheek
Copy link
Member

As a data point, this is a Linux-only quirk (probably for backwards-compat in glibc). Bionic, FreeBSD, and Darwin always enable large file support. However, even on Linux if you disable large-file support, stat and friends will try to return 64-bit inode values and you will get EOVERFLOW.

@jpakkane
Copy link
Member

Probably the only OS that does 32 bit file sizes is Windows and if you are using Windows native system calls you already know what you are doing.

@nirbheek
Copy link
Member

Yes, on Windows you have to use _fseeki64 because fseek only takes a long: https://msdn.microsoft.com/en-us/library/75yw9bf3.aspx

The thing is on Linux you need to still set compiler flags to get large-file support; specifically you need to set _FILE_OFFSET_BITS and _LARGE_FILES. The details are in the AC_SYS_LARGEFILE macro. Without this, open() returns EOVERFLOW when you try to open large files with gstreamer on 32-bit Linux.

@tp-m
Copy link
Member Author

tp-m commented Nov 12, 2016

For what it's worth AC_SYS_LARGEFILE also adds

#ifndef _DARWIN_USE_64_BIT_INODE
#define _DARWIN_USE_64_BIT_INODE
#endif

to config.h (unconditionally, mind). Don't know if it's still needed these days or not.

@nirbheek
Copy link
Member

Actually, I now remember that all versions of OS X are now 64-bit so it doesn't matter for Darwin.

@tp-m
Copy link
Member Author

tp-m commented Dec 16, 2016

So what shall we do about this? I think just enabling it unconditionally based on platform is fine, with or without checking. But where does this go implementation-wise? just add to global args?

@nirbheek
Copy link
Member

I think unconditionally based on platform is fine without checking. We can change it if people complain about their precious ancient-kernel-or-libc broken 32-bit platforms.

Implementation-wise, it would go in global C and C++ args, but I haven't decided exactly which part of the code is best for adding it. For instance, we definitely want these to also be added when building C code generated by Vala, and also when doing compiler checks.

@nirbheek nirbheek modified the milestones: 0.37.0, 0.38.0 Dec 18, 2016
@nirbheek nirbheek modified the milestones: 0.38.0, 0.39.0 Feb 1, 2017
@nirbheek nirbheek modified the milestones: 0.39.0, 0.40.0 Mar 9, 2017
jpakkane pushed a commit that referenced this issue Mar 10, 2017
On 32-bit Linux and BSD, all libcs support this. Musl always enables it,
and UClibc behaves like Glibc unless it's built without large file
support (which is a terrible idea).

http://wiki.musl-libc.org/wiki/FAQ#Q:_do_i_need_to_define_LARGEFILE64_SOURCE_to_get_64bit_off_t_.3F
https://git.uclibc.org/uClibc/tree/include/features.h#n326

macOS now only ships 64-bit, so this is irrelevant there.

32-bit Windows and older versions of Bionic do not have transparent
large file support and you must use lseek64, etc there, so this won't
affect those. Newer Bionic versions behave like Glibc in theory.

https://msdn.microsoft.com/en-us/library/1yee101t.aspx
http://code.google.com/p/android/issues/detail?id=64613

Includes a linuxlike test for this.

Closes #1032
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants