Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PDL 2.083 build errors #438

Closed
perlboy-emeritus opened this issue May 15, 2023 · 14 comments
Closed

PDL 2.083 build errors #438

perlboy-emeritus opened this issue May 15, 2023 · 14 comments

Comments

@perlboy-emeritus
Copy link

perlboy-emeritus commented May 15, 2023

I've been trying for days to upgrade PDL from 2.021 to 2.083, and so far, nothing, including advice from ej_zg has resolved my issue, a segmentation fault in compilation of pdlcore.c, to wit:

cc -c  "-I/Users/carly/.cpanm/work/1684165713.4672/PDL-2.083/Basic/Core" -fno-common -DPERL_DARWIN -mmacosx-version-min=10.13 -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -I/opt/local/include -DPERL_USE_SAFE_PUTENV -O3   -DVERSION=\"2.083\" -DXS_VERSION=\"2.083\"  "-I/Users/carly/perl5/perlbrew/perls/perl-5.30.2/lib/5.30.2/darwin-2level/CORE"  -DPDL_PTHREAD  pdlcore.c
clang: error: unable to execute command: Segmentation fault: 11
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg

I am running Perl

$ perl -v
This is perl 5, version 30, subversion 2 (v5.30.2) built for darwin-2level
(with 1 registered patch, see perl -V for more detail)

on macOS 10.13.6. Per ej_zg "The segfault you show below is from a bug in that version of clang. Upgrade xcode and it should work.

libdf is part of HDF4, and the Alien package exists to wrap knowledge about that. PDL should install and work fine if HDF4 isn't present. "

I was running Xcode 9.2. I've upgraded command line tools twice, to 9.4.1 and then 10.1 but PDL refuses to build.

$ clang --version
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

When I upgrade Alien my package is up to date, but I am still missing libdf.a, and without that library HDF4 will just not build. I am now without options as to how to get this or any older versions (if I could even find them) of PDL to build. Please don't suggest upgrading macOS just to get the latest PDL. That would cost me days of work fetching, building and/or patching other essential software I must have. It ought not to be this difficult to do a simple version upgrade.

Any help would be much appreciated.

@perlboy-emeritus
Copy link
Author

Since posting my build report I found HDF-4.2.16 for Apple clang 10,0.0 and gfortran 6.3.0 at https://portal.hdfgroup.org/display/support/HDF+4.2.16#files and was able to successfully execute the shell script HDF-4.2.16-Darwin.sh. This script seemed to execute without error and built a huge tree of files including libdf.a. The question now is, what do I do with this stuff to make it available to cpanm building PDL? In other words, where do I put this stuff, or a link to it OTHER THAN IN THE macOS SYSTEM DIRECTORIES. I depend on perlbrew to keep my private, non-Apple Perl implementations current. Apple owns and uses the Perl that came with the OS and I prefer to not touch it, if possible.

@mohawk2
Copy link
Member

mohawk2 commented May 20, 2023

In order to make this work, you would need to have Alien::HDF4 return suitable C compiler flags for the location you've got HDF installed in. A "proper" Alien package these days, built on top of Alien::Build, would make this very easy, using a "share install".

Something like an explanation of share vs system can be seen at the docs' most relevant section. If you were minded to make a full, "proper" Alien::Build-using Alien::HDF4 instead of the hacked-together thing I made, that document will help you do so, and would probably not be hard. It would also be a great contribution! If you are so minded, please put it together and make a pull-request on the relevant repository.

For your purposes, you could simply edit your installed Alien::HDF4::Install::Files module (locate it with perldoc -l Alien::HDF4::Install::Files) to return appropriate things.

Keeping up to date with the latest MacOS might cause you some short-term disruption, but would avoid nearly all of the problems you are experiencing, and would also close various security holes. I will not support an old, insecure, obsolete version of MacOS, nor of clang, but I will be pleased to accept patches that make those work.

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented May 20, 2023 via email

@PerlSAP
Copy link

PerlSAP commented May 20, 2023

Interesting find re: the C standard macro length.

However, wouldn't a truly open-source-informed approach be to submit a patch/ pull request that fixes the macro in question? (I.e. a couple of appropriately parametrised sub-macros that get incorporated in the currently huge one ought to do it. Hint hint).

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented May 20, 2023 via email

@mohawk2
Copy link
Member

mohawk2 commented May 21, 2023

I am sure it has taken less time for me to write and run this code than it took @perlboy-emeritus to type in his humble-brag life story (whose relevance to this issue I cannot fathom):

$/=undef;
$_=<>;
push @found, [$1,$2]
  while /
    ^
    [ \t]*
    \#[ \t]*
    define[ \t]+
    ([a-zA-Z][^\(\s]+)
      [ \t]*\(.*?\)[ \t]*
    (.*?[^\\]\n)
  /gmsx;
#use Data::Dumper; print Dumper \@found;
print "$_->[0]: ", length($_->[1]), "\n" for @found;

which when run on pdlcore.c gives:

GEN_PDL_BARF_OR_WARN_I_STDARG: 298
INNERLOOP_X: 1333
PDL_KLUDGE_COPY_X: 4884
PDL_SETAV_X: 3131

@mohawk2
Copy link
Member

mohawk2 commented May 21, 2023

And the deep, cross-platform refactoring of the relevant macro by removing some whitespace has now reduced it to 3957 characters, which took <15 mins.

@mohawk2
Copy link
Member

mohawk2 commented May 21, 2023

Now released as 2.084.

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented May 21, 2023 via email

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented May 21, 2023 via email

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented May 22, 2023 via email

@mohawk2
Copy link
Member

mohawk2 commented May 29, 2023

A real software engineer would:

  • know my Perl expression does not modify anything
  • realise C macro length includes whitespace
  • know that CPP operates before any lexing stage, rendering the "whitespace token" points above irrelevant
  • know how to use git to see different versions of files
  • be capable of capturing exactly the before-and-after versions of any files they modified in building/installing a piece of software
  • strip any such changes down to the bare minimum to achieve the change between success and failure
  • be able to submit such changes as a pull request

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented Jun 13, 2023 via email

@perlboy-emeritus
Copy link
Author

perlboy-emeritus commented Jun 14, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants