use64bitint is mandatory for NetBSD. #18788
Conversation
Forcibly enable use64bitint for NetBSD 4.0 and later, where ino_t is uint64_t. Otherwise, miniperl cannot open files whose inode numbers are not within 32-bit integer. This results in random build failures.
I very much doubt that we should adopt the approach we are advocating. I don't claim any particular expertise with NetBSD. I'm more familiar with FreeBSD and OpenBSD. But we have a long history of successfully smoke-testing Perl 5 blead on NetBSD in a variety of configurations without specifying anything one way or the other about For you to convince us we would need some more evidence of the Thank you very much. |
|
Thank you very much for your quick reply. The problem is tracked as Perl successfully builds on NetBSD for most cases; for LP64 architectures, and for ILP32 architectures with real disk drives. For ILP32 architectures, this is because inode number is normally within 32-bit integer for real storage. However, typically for tmpfs on (*) NetBSD provides |
What happens when perl is being built on a 32 bit architecture under netbsd with this change? Would it be "nothing"? Just thought I'd see some check on the actual architecture. NetBSD 4.0 is from 2008, so it's reasonable to consider this case. |
|
With this patch, Perl is configured as if Although Please tell me how, if I can/should do some regression tests. |
|
I'm curious about what's going wrong at the system level, is stat() failing? -Duselargefiles (typically on by default) should be preventing that, and pp_stat should be handling large inode numbers sanely (either storing as a double if they're exactly representable, or as a string). The strange @inc is the problem, and I've reproduced it here, for anyone else:
|
|
Thank you very much for reproducing the problem as well as kind explanation. As you pointed out, a large inode numbers are stored as strings. However, they are compared as integers here: https://github.com/Perl/perl5/blob/blead/dist/PathTools/Cwd.pm#L278 This is impossible for ILP32 architectures when use64bitint is disabled. As a minimal test case, this does not work properly: Here, values of |
|
The issue is that this isn't just a NetBSD issue - other systems have large inode numbers too: So we need to fix this generally, not just for NetBSD. The simplest fix is for Cwd (and File::Find and Porting/bisect.pl, and probably a lot of CPAN, including IO::Async::File, AnyEvent, but not File::Path, File::Temp) to start using eq instead of == for comparing inode numbers. Another solution might be for us to return a Math::BigInt or some other overloaded object for large inode numbers (or for all inode numbers to avoid surprises), |
|
Thank you for your comment. At least Perl successfully builds on tmpfs with this patch: https://gist.github.com/rokuyama/499f7bd1c02630d1e0859ebf020d54ba by which inode numbers are compared as strings. |
On a 32-bit system with 64-bit inodes, recent-ish versions of perl will return very large inode numbers as strings, and their numeric value may be too large to be exactly represented as an NV. This means comparing such inodes numerically may return them as equal even when the actual inode number is different. See Perl/perl5#18788 for a case where this has caused a problem.
|
Still need the MM changes |
On a 32-bit system with 64-bit inodes, recent-ish versions of perl will return very large inode numbers as strings, and their numeric value may be too large to be exactly represented as an NV. This means comparing such inodes numerically may return them as equal even when the actual inode number is different. See Perl/perl5#18788 for a case where this has caused a problem.
PR pkg/55997 Internal stat() function for perl stores inode number as string, if it cannot be represented by host's integer. However, unfortunately, some components compare them as integer. Therefore, if 64-bit integers are not supported, files cannot be handled, whose inode number is larger than UINT32_MAX. Usually, inode numbers on real filesystems are well below UINT32_MAX. But, inode numbers larger than UINT32_MAX are assigned for tmpfs on LP64 kernels. This results in build failures for perl on COMPAT_NETBSD32 if working directory is tmpfs, and perl-64bitint and friends are not specified. Now, inode numbers are compared as string, which works just fine even if 64-bit integers are not supported. Cherry-picked from upstream. See Perl/perl5#18788 and related pull-requests for more details.
Am I correct in thinking that if an ExtUtils-MakeMaker 7.63 is released to CPAN, we can then proceed with this p.r.? Thank you very much. |
No, this PR is obsoleted by the #18798 and MM 7.63 to-be changes. This PR is effectively now an issue, but github has no mechanism to convert a PR to an issue. |
Can you or @rokuyama or someone else propose some language to open a new issue about this? |
|
I've pulled all the information I can discover into #19489, so I think this PR can be closed now. I'll do that in a day or two if nobody dissents. |
What specific steps, if any, should we then take with respect to NetBSD? |
No specific steps should be required, it is a general problem; but see also above where "netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue on May 29, 2021" - I expect that resolved the immediate problem that prompted this PR in the first place. As Tony said, "this PR is effectively now an issue, but github has no mechanism to convert a PR to an issue". |
|
Closing per discussion; thanks @rokuyama for bringing this to our attention. |
5 participants
Forcibly enable use64bitint for NetBSD 4.0 and later, where ino_t is uint64_t.
Otherwise, miniperl cannot open files whose inode numbers are not within
32-bit integer. This results in random build failures.