OSX El Capitan PCP Installation failure

[tallpsmith]

I upgraded my OSX to the latest El Capitan release, noticed pmchart was no longer on the path, so tried to just reinstall it, but it failed.

I've attached the full install log, I suspect permissions of some PCP directories have been borked by some part of the El Capitan upgrade.

I'll see if I can work out a hack to fix it, but it may need some changes to the PCP installer ?

[tallpsmith]

Attaching a installer file.

[tallpsmith]

damn, upload won't work for some reason, I've pasted it as a gist:

https://gist.github.com/tallpsmith/e9a92083116dbcfcfd29

[natoscott]

Looks like it all goes pear-shaped here (EPERM?) ... very strange, because the uninstall of the previous version seems to rmdir that path successfully.

Oct 14 09:07:43 paul installd[373]: ./postupgrade: /bin/mkdir -p /usr/include/pcp
Oct 14 09:07:44 paul installd[373]: Oct 14 09:07:43 paul installd[373]: ./postupgrade: /bin/mkdir -p /usr/include/pcp
Oct 14 09:07:44 paul installd[373]: ./postupgrade: mkdir: /usr/include/pcp: Operation not permitted
Oct 14 09:07:44 paul installd[373]: ./postupgrade: install-pcp: Error: Unable to create dir /usr/include/pcp
Oct 14 09:07:44 paul installd[373]: ./postupgrade: postupgrade: Install Failed./postupgrade: mkdir: /usr/include/pcp: Operation not permitted
Oct 14 09:07:44 paul installd[373]: ./postupgrade: install-pcp: Error: Unable to create dir /usr/include/pcp
Oct 14 09:07:44 paul installd[373]: ./postupgrade: postupgrade: Install Failed

[tallpsmith]

so, after the El Capitan upgrade, /usr/include does not exist... like... at all.

[ ~ ]$ ll /usr/ X11/ adic/ bin/ lib/ libexec/ local/ sbin/ share/ standalone/

[tallpsmith]

weirdly.. Even as root I cannot create a directory in /usr/. Queue the X-Files..

[tallpsmith]

and here it is... https://en.wikipedia.org/wiki/System_Integrity_Protection

[natoscott]

Interesting read, thanks Paul - it sounds like we should start building PCP on Mac such that binaries, headers, etc end up in /usr/local rather than /usr (from that article anyway). OOC, is /usr/local/bin on the default path on El Capitan, or would such a move break things?

[tallpsmith]

I can't say I'm 100% sure because I've probably modified my local account, but my sudo su - on OSX I checked the root path and it looks like this:

paul:~ root# echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

[pcpemail]

Random two cents: I had a look at this out of curiosity and found that the
default PATH seems to be populated from /etc/paths and /etc/paths.d:

powell:~ serhei$ ls -l /etc/paths

-rw-r--r-- 1 root wheel 45 22 Aug 17:35 /etc/paths

powell:~ serhei$ ls -l /etc/paths.d

total 16

-rw-r--r-- 1 root wheel 13 11 Aug 2014 40-XQuartz

-rw-r--r-- 1 root wheel 22 27 Jul 22:13 TeX

powell:~ serhei$ cat /etc/paths.d/40-XQuartz

/opt/X11/bin

powell:~ serhei$ cat /etc/paths

/usr/local/bin

/usr/bin

/bin

/usr/sbin

/sbin

So there seem to be a lot of tricks available for software to put itself on
the $PATH.

I know for a fact that I installed XQuartz (which installs under /opt
rather than /usr/local) after upgrading to El Capitan. Thus SIP blocks
anything from being added to /usr (outside of /usr/local), but /etc is fair
game.

There are also package managers for OS X, namely MacPorts and Homebrew.
They have their own conventions that might be worth looking into (either to
imitate them, or possibly to avoid conflicting with what they do).

-- Serguei Makarov

On Thu, Oct 15, 2015 at 9:49 PM, Paul Smith notifications@github.com
wrote:

I can't say I'm 100% sure because I've probably modified my local account,
but my sudo su - on OSX I checked the root path and it looks like this:

paul:~ root# echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

—
Reply to this email directly or view it on GitHub
#48 (comment)
.

---

pcp mailing list
pcp@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/pcp

[tallpsmith]

I tried a few things, having found that homebrew on OSX is also hosed due to SIP, I found this document:

https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/El_Capitan_and_Homebrew.md

The gist being to change the ownership of /usr/local, this I think seems to get a bit further when installing PCP except it still can't create /usr/include.

from the installer log:

Oct 21 07:43:09 paul installd[489]: ./postupgrade: mkdir: /usr/include/pcp: Operation not permitted
Oct 21 07:43:09 paul installd[489]: ./postupgrade: install-pcp: Error: Unable to create dir /usr/include/pcp
Oct 21 07:43:09 paul installd[489]: ./postupgrade: postupgrade: Install Failed

I can't create this directory even with sudo.

What's needed in /usr/include exactly?

[natoscott]

The PCP API header files (like pmapi.h, pmda.h, etc) are put there for people building tools using C/C++. They can safely transition to being below /usr/local/include though.

[natoscott]

Verified fix in upstream master branch, El Capitan with SIP enabled will be able to build PCP from release pcp-3.10.9 onward.

[tallpsmith]

Nathan while we wait for further work on #52 could you provide a link to a DMG from your OSX build for 3.10.9 ?

I could really do with that to use and share around here in Aconex.

[natoscott]

Hi Paul, I don't have a working Qt installation on this build machine atm, so you probably don't want this build. I'll make sure its resolved before next release though (2 weeks).

Also, no need to wait for me on the PackageMaker vs pkgbuild problem -- if someone else could investigate the necessary changes to the "build/mac/build-installer" script to support pkgbuild (or even just provide the right incantation/options for pkgbuild use), that would be extremely helpful.

[tallpsmith]

I actually tried to get pkgbuild running but at a loss to know what it is that it needs too. I wish we could pair on this together!

I'm not even sure if I have a Qt installation locally either. I guess I would like to know, and help document what is required to build on OSX, right now it doesn't work for me, and for others I guess, so whatever machine you are building the OSX images for is a single point of failure, because it will be tricky to setup another machine to create any future builds if we are not able to correctly document or script the steps needed to install the necessary bits?

I will endevour to keep poking about, and I will reach out to IRC too when I get a chance to work on this to see if we can collaborate together. I'll use other Engineers here at Aconex to help test the steps needed too so we can be sure we have a repeatable series of steps.

[kmcdonell]

Does not help El Capitan per se, but I now have a successful Mac OS X 10.8.0 build and a recipe.
Look for README.osx at the top of the PCP git tree once my commit percolates up to the official repo.

[tallpsmith]

Hi Ken! I see these commits with interest.

I dug a little into your README, and notice that we might have a simpler way. I notice there is still reference to MacPorts, however that system has generally been out of favour for new, and replaced with something called HomeBrew:

http://brew.sh/

I think you could get away without needing the manual installation steps if you use Homebrew by using:

brew install libmicrohttpd pixman cairo libpng xz

Once you have Homebrew installed.

I am still having an issue however, I'll post the logs here shortly.

[tallpsmith]

Here's the build error from the non-verbose running:

[ kenj-pcp ](master)$ ./Makepkgs

== Cleaning build tree for packaging build
Wrote: /Users/psmith/dev/pcp/kenj-pcp/build/tar/pcp-3.10.9.src.tar.gz

== Preparing fresh build tree for packaging build

== Configuring pcp, log is in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp (--prefix=/usr/local --sysconfdir=/etc --localstatedir=/var)

== Building pcp, log is in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp
Make default_pcp failed, see log in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp
      pmgraphite_respond_render_gfx(MHD_Connection*, http_params const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) in pmgraphite.o
  "_cairo_surface_write_to_png_stream", referenced from:
      pmgraphite_respond_render_gfx(MHD_Connection*, http_params const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) in pmgraphite.o
  "_cairo_text_extents", referenced from:
      pmgraphite_respond_render_gfx(MHD_Connection*, http_params const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) in pmgraphite.o
ld: symbol(s) not found for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [pmwebd] Error 1
make[1]: *** [default_pcp] Error 2
make: *** [default_pcp] Error 2

[tallpsmith]

Maybe that's a cairo version compatability issue, here's the version Homebrew has installed:

[ ~ ]$ brew info cairo
cairo: stable 1.14.4 (bottled)
Vector graphics library with cross-device output support
http://cairographics.org/
/usr/local/Cellar/cairo/1.12.14 (106 files, 7.4M)
  Built from source
/usr/local/Cellar/cairo/1.12.16 (105 files, 7.6M)
  Built from source
/usr/local/Cellar/cairo/1.12.16_1 (105 files, 7.6M)
  Poured from bottle
/usr/local/Cellar/cairo/1.14.0 (106 files, 6.4M)
  Poured from bottle
/usr/local/Cellar/cairo/1.14.2 (116 files, 6.7M)
  Poured from bottle
/usr/local/Cellar/cairo/1.14.2_1 (111 files, 6.1M)
  Poured from bottle
/usr/local/Cellar/cairo/1.14.4 (112 files, 6.1M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/cairo.rb
==> Dependencies
Build: xz ✔, pkg-config ✔
Required: freetype ✔, fontconfig ✔, libpng ✔, pixman ✔, glib ✔
==> Options
--universal
    Build a universal binary
--with-x11
    Build with x11 support

[tallpsmith]

other package version:

[ ~ ]$ brew info xz libpng pixman libmicrohttpd
xz: stable 5.2.2 (bottled)
General-purpose data compression with high compression ratio
http://tukaani.org/xz/
/usr/local/Cellar/xz/5.0.4 (58 files, 1.5M)
/usr/local/Cellar/xz/5.0.5 (58 files, 1.5M)
  Built from source
/usr/local/Cellar/xz/5.0.6 (58 files, 1.5M)
  Poured from bottle
/usr/local/Cellar/xz/5.0.7 (58 files, 1.5M)
  Poured from bottle
/usr/local/Cellar/xz/5.2.0 (59 files, 1.7M)
  Poured from bottle
/usr/local/Cellar/xz/5.2.1 (59 files, 1.7M)
  Poured from bottle
/usr/local/Cellar/xz/5.2.2 (59 files, 1.7M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/xz.rb
==> Options
--universal
    Build a universal binary

libpng: stable 1.6.19 (bottled), HEAD
Library for manipulating PNG images
http://www.libpng.org/pub/png/libpng.html
/usr/local/Cellar/libpng/1.5.13 (14 files, 1.0M)
/usr/local/Cellar/libpng/1.5.14 (15 files, 1.0M)
/usr/local/Cellar/libpng/1.5.17 (15 files, 1.0M)
  Poured from bottle
/usr/local/Cellar/libpng/1.5.18 (15 files, 1.0M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.10 (17 files, 1.3M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.12 (17 files, 1.2M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.13 (17 files, 1.2M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.14 (17 files, 1.2M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.15 (17 files, 1.3M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.16 (17 files, 1.3M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.17 (17 files, 1.2M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.18 (17 files, 1.2M)
  Poured from bottle
/usr/local/Cellar/libpng/1.6.19 (17 files, 1.2M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/libpng.rb
==> Dependencies
Build: xz ✔
==> Options
--universal
    Build a universal binary
--HEAD
    Install HEAD version

pixman: stable 0.32.8 (bottled)
Low-level library for pixel manipulation
http://cairographics.org/
/usr/local/Cellar/pixman/0.28.2 (11 files, 1.1M)
  Built from source
/usr/local/Cellar/pixman/0.30.0 (11 files, 1.3M)
  Built from source
/usr/local/Cellar/pixman/0.30.2 (11 files, 1.3M)
  Built from source
/usr/local/Cellar/pixman/0.32.4 (11 files, 1.2M)
  Built from source
/usr/local/Cellar/pixman/0.32.6 (11 files, 1.4M)
  Poured from bottle
/usr/local/Cellar/pixman/0.32.8 (11 files, 1.2M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/pixman.rb
==> Dependencies
Build: pkg-config ✔
==> Options
--universal
    Build a universal binary

libmicrohttpd: stable 0.9.42 (bottled)
Light HTTP/1.1 server library
https://www.gnu.org/software/libmicrohttpd/
/usr/local/Cellar/libmicrohttpd/0.9.42 (14 files, 712K) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/libmicrohttpd.rb
==> Options
--universal
    Build a universal binary
--with-ssl
    Enable SSL support

[kmcdonell]

Paul,
Given ...
Option A: works with out of favor MacPorts, and
Option B: does not work with flavor du jour HomeBrew
it is an easy decision for me ... 8^)

Your build is failing because the libmicrohttpd that the PCP build is linking against is for the wrong arch. Is it possible that you have multiple libmicrohttpd's installed ... one from HomeBrew and one the PCP build is finding?

[tallpsmith]

:-P

You are of course correct Ken, but in the spirit of hoping that we get more OSX users actually building this thing, it may be a good idea to 'bend a little' to the momentum of where the OSX community is. The more esoteric the environment, the less likely it will be for anyone to bother.. Loads of Devs will have Homebrew; they'll be up and going quicker if we do a bit more following.. :)

Anyway, in the spirit of Borot, I bring you "GREAT SUCCESS". I have discovered the need to use Homebrew to install the dependencies with the --universal option which builds cross-architecture binaries.

So what I did was:

brew uninstall --force libmicrohttpd pixman cairo libpng xz
 brew install --universal libmicrohttpd pixman cairo libpng xz

Then I get a cleanly built tree:

[ kenj-pcp ](master)$ ./Makepkgs

== Cleaning build tree for packaging build
Wrote: /Users/psmith/dev/pcp/kenj-pcp/build/tar/pcp-3.10.9.src.tar.gz

== Preparing fresh build tree for packaging build

== Configuring pcp, log is in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp (--prefix=/usr/local --sysconfdir=/etc --localstatedir=/var)

== Building pcp, log is in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp

== Packaging pcp, log is in /Users/psmith/dev/pcp/kenj-pcp/Logs/pcp
Wrote: /workspace_intellij/pcp/kenj-pcp/pcp-3.10.9/build/tar/pcp-3.10.9-1.tar.gz

Of course I still can't see a DMG being built anywhere, but I'll take it as a success that a clean checkout of your tree with the Homebrew install is working cleanly, this is awesome. Thanks!

[tallpsmith]

so the configure is still looking for PackageMaker, which is just simply not available for later versions of OSX anymore (I really think it's stupid of Apple to do this)

From the ./Makepkgs --verbose

...
checking for PackageMaker...  not found!
configure: WARNING: PackageMaker not found, mac packages will not be made
...

So somehow need to work out how to change PackageMaker -> pkgbuild, I can't yet make head or tails of it as yet, because I'm not sure what PackageMaker was doing originally..

[tallpsmith]

ok so getting somewhere, I realise now the difference between a pkg and a dmg, i was getting too far ahead of myself, if I manually run these steps after Makepkg within the mac build area I get something I can double click on and run the 'installer':

pkgbuild --root . --identifier pcp --version 3.10.9  /tmp/foo.pkg

This creates the foo.pkg that appears to look reasonable!

hdiutil will be used to bundle it into a DMG, that bit is probably working if the PackageMaker stuff actually completes.

So the last remaining trick I think is that the identifier and other options from the man page of pkgbuild (like --install-location probably) need to be worked out.

Also, late breaking, the pkg does install nicely, but I don't think the contents are actually supposed to be the directory I thought it was..

[tallpsmith]

Just documenting where I got to in the end before Beer-O'Clock, I started trying to hack the build-installer file which currently relies on PackageMaker, and tried something like:

# PackageMaker returns non-zero value even if it succeeds.
# PackageMaker must be run by the user logged into the localhost desktop.
#
_run pkgbuild $HERE/$PKG  \
    --scripts $HERE/installer-files  \
    -i $HERE/info -d $HERE/description $HERE/$PKG&
wait

This doesn't really work... The --scripts is correct I think, but what is actually missing is the actual contents of PCP binaries. Like where is the binary for pmlogger ? I can't find it locally..

I wonder if there are more errors going on during the build where it's literally not building anything still.. All I have for the Mac build is a directory shell of Mac-y things needed, but not the actual PCP binaries... Unless they're somewhere I am too stupid to see.

[natoscott]

Hi Paul - good progress! A few random hints that might help ...

| ... what is actually missing is the actual contents of PCP binaries.
| Like where is the binary for pmlogger ? I can't find it locally..

That pcp-3.10.9/build/mac/idb file is the key here. It tells the packaging tool where to find the built binaries and where theshould up when installed. You should see a pcp-3.10.9/src/pmlogger/pmlogger binary, which that idb file references (and maps to a usr/local/bin/pmlogger end point).

| I wonder if there are more errors going on during the build where it's literally not building anything still..

When you use Makepkgs, there is a ./Logs/pcp file that will give you all the details about what got built, what got packaged, etc... skip to the end for any errors.

| So somehow need to work out how to change PackageMaker -> pkgbuild, I can't yet make
| head or tails of it as yet, because I'm not sure what PackageMaker was doing originally..

We'll want to get to the point where the configure.ac script detects pkgbuild and passes that down to the build/mac directory for use in that part of the build. We may want a separate build-installer alike script for pkgbuild if its a very different packaging process to PackageMaker.

[tallpsmith]

pcp-3.10.9/build/mac/idb is literally not there, I will have to dig in to
why.

On Fri, Dec 4, 2015 at 5:31 PM Nathan Scott notifications@github.com
wrote:

Hi Paul - good progress! A few random hints that might help ...

| ... what is actually missing is the actual contents of PCP binaries.
| Like where is the binary for pmlogger ? I can't find it locally..

That pcp-3.10.9/build/mac/idb file is the key here. It tells the packaging
tool where to find the built binaries and where theshould up when
installed. You should see a pcp-3.10.9/src/pmlogger/pmlogger binary, which
that idb file references (and maps to a usr/local/bin/pmlogger end point).

| I wonder if there are more errors going on during the build where it's
literally not building anything still..

When you use Makepkgs, there is a ./Logs/pcp file that will give you all
the details about what got built, what got packaged, etc... skip to the end
for any errors.

| So somehow need to work out how to change PackageMaker -> pkgbuild, I
can't yet make
| head or tails of it as yet, because I'm not sure what PackageMaker was
doing originally..

We'll want to get to the point where the configure.ac script detects
pkgbuild and passes that down to the build/mac directory for use in that
part of the build. We may want a separate build-installer alike script for
pkgbuild if its a very different packaging process to PackageMaker.

—
Reply to this email directly or view it on GitHub
#48 (comment)
.

[tallpsmith]

I get a sense that maybe my make is not the same as everyones, here's the output of that log file.

The text:

=== ganglia2pcp ===
make[3]: Nothing to be done for `default_pcp'.

and many many many of that ilk is possibly telling...

pcp.txt

[tallpsmith]

btw, I think this issue needs to be re-opened (it's closed - but I can't build)..

I am getting a bit further, the idb file only works if the PackageMaker or it's ilk is discovered, I managed to hack in an else block in the PackageMaker section to seek out the pkgbuild and use that for package_maker which now gets me the missing idb.

I'll see if I can have the build-installer further to check for pkgbuild a bit more.

[natoscott]

| btw, I think this issue needs to be re-opened (it's closed - but I can't build)..

Yep, sure - done. There are actually a couple of known issues remaining here...

| [...] only works if the PackageMaker or it's ilk is discovered

This is the first problem - "no build support for /usr/bin/pkgbuild". After that there is a second issue that the GUI tools cannot successfully build with Qt5 yet, and Qt4 cannot be installed on El Capitan.

Some work has been done on the second issue, but there's a fair way to go yet (there's a cascading issue that we need Qwt6.1, which supports Qt5, but pmchart needs work to adapt to the interface changes there).

It's never easy is it?

[tallpsmith]

Ok I have forked and committed one change that I have managed to get to work on El Capitan using Homebrew (brew install... see above), and the use of pkgbuild instead of PackageMaker.

tallpsmith/pcp@227d392

I make no claims that this is the right way to do it, I'm sure if we want to have dual PackageMaker/pkgbuild support much more work needs to be done.

But the bloody thing now works on my OSX so I'll take it for now!:

[ ~ ]$ pminfo -f -h au1appa.acx hinv.ncpu

hinv.ncpu
    value 48

[tallpsmith]

Incidentally @natoscott your reference to QT and El Capitan, Homebrew seems to have ways to install qt and qt5 and when looking at qt it looks to be version 4:

[ ~ ]$ brew info qt
qt: stable 4.8.7 (bottled), HEAD
Cross-platform application and UI framework
https://www.qt.io/
Not installed
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/qt.rb
==> Dependencies
Required: openssl ✔
Optional: d-bus ✔, mysql ✔, postgresql ✘
==> Options
--universal
    Build a universal binary
--with-d-bus
    Build with d-bus support
--with-developer
    Build and link with developer options
--with-docs
    Build documentation
--with-mysql
    Build with mysql support
--with-postgresql
    Build with postgresql support
--with-qt3support
    Build with deprecated Qt3Support module support
--HEAD
    Install HEAD version
==> Caveats
We agreed to the Qt opensource license for you.
If this is unacceptable you should uninstall.

I'm going to try a --universal install of this and see if pmchart starts magically building as part of this, I'm not expecting it to but will report back if it does.

Where did you find that QT won't work on El Capitan ?

[natoscott]

| Homebrew seems to have ways to install qt and qt5 and when looking at qt it looks to be version 4:

Oh, that's interesting and that might well work (its a rebuild, I believe, and not binaries from qt.io).

| Where did you find that QT won't work on El Capitan ?

I attempted to install both the current version (5.x) and the latest 4.8.x from qt.io - the latter fails to install, the former is a-OK. AIUI 5.x is the tested-on-El-Capitan version (by the folk releasing Qt).

[tallpsmith]

I'm getting closer, looks like a non --universal build of qt seems to install, and a minor hack of configure to include the path /usr/local/bin when looking for qmake seems to find QT ok, lets cross fingers.

[tallpsmith]

close, but not breaking out the cigars just yet:

cp: /Library/Frameworks//usr/local/opt/qt/lib/QtSvg.framework/Versions/4/QtSvg: No such file or directory

[tallpsmith]

Looks like it is there as part of Homebrew install, but in this location:

/usr/local/Cellar/qt/4.8.7_1/lib/QtSvg.framework/Versions/4/QtSvg