Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MacSDK] pkg-config is 32-bit only #13804

Closed
alexischr opened this issue Apr 2, 2019 · 22 comments

Comments

@alexischr
Copy link
Member

@alexischr alexischr commented Apr 2, 2019

Something has changed in how pkg-config is built, and instead of the expected lipo-ed 32-bit/64-bit Mach-O executable, we only get the 32-bit build.

For example:

The pkg-config tarball/version hasn't changed so this is possibly because of XCode updates on the bots.

alexischr added a commit that referenced this issue Apr 2, 2019
alexischr added a commit that referenced this issue Apr 3, 2019
akoeplinger added a commit that referenced this issue Apr 3, 2019
akoeplinger added a commit that referenced this issue Apr 3, 2019
akoeplinger added a commit that referenced this issue Apr 3, 2019
akoeplinger added a commit that referenced this issue Apr 8, 2019
@akoeplinger

This comment has been minimized.

Copy link
Member

@akoeplinger akoeplinger commented Apr 8, 2019

Fixed by #13805

@akoeplinger akoeplinger closed this Apr 8, 2019
jonpryor added a commit to jonpryor/xamarin-android that referenced this issue Apr 8, 2019
Fixes: mono/mono#13804
Fixes: mono/mono#13834

Don't do thread state transitions in `mono_dl_fallback_register()`.
@akoeplinger

This comment has been minimized.

Copy link
Member

@akoeplinger akoeplinger commented Apr 26, 2019

I just saw the same issue with this mono which was built after the fix:

Mono JIT compiler version 6.0.0.184 (2019-02/3a8f93b9a47 Thu Apr 18 09:17:09 EDT 2019)
@akoeplinger akoeplinger reopened this Apr 26, 2019
@alexischr

This comment has been minimized.

Copy link
Member Author

@alexischr alexischr commented May 6, 2019

I have so far been unable to reproduce. @directhex do you have any insight on this issue or how to debug it? I am still not sure - it may be a bockbuild issue or an issue with pkg-config's build.

@directhex

This comment has been minimized.

Copy link
Contributor

@directhex directhex commented May 9, 2019

It seems to be doing the right thing, in the build logs?

e.g.

07:23:14  pkg-config 0.27
07:23:14  
07:23:14  pkg-config->start_build:
07:23:14  	pkg-config 0.27 <http://pkg-config.freedesktop.org/releases/pkg-config-0.27.tar.gz> md5: 3a4c9feab14b6719afd8904945d9b4e4)
07:23:14  
07:23:14  pkg-config->do_build:
07:23:14  	Building (arch: darwin-32)
07:23:14  
07:23:14  	@	./configure --prefix="/Library/Frameworks/Mono.framework/Versions/6.0.0" --with-internal-glib --build=i386-apple-darwin13.0.0 --disable-dependency-tracking --cache-file=�[1m@�[0mexternal/bockbuild/builds/pkg-config-darwin-32.cache ac_cv_func_basename_r=no ac_cv_func_clock_getres=no ac_cv_func_clock_gettime=no ac_cv_func_clock_settime=no ac_cv_func_dirname_r=no ac_cv_func_getentropy=no ac_cv_func_mkostemp=no ac_cv_func_mkostemps=no ac_cv_func_timingsafe_bcmp=no
07:24:03  	@	make -j4
07:24:07  	@	make install DESTDIR=�[1m@�[0mexternal/bockbuild/scratch/pkg-config-x86.install
07:24:07  
07:24:07  darwin->postprocess:
07:24:07  	stage_textfiles: 0 items
07:24:07  	stage_binaries: 1 items
07:24:08  
07:24:08  pkg-config->do_build:
07:24:08  	Building (arch: darwin-64)
07:24:08  
07:24:08  	@	./configure --prefix="/Library/Frameworks/Mono.framework/Versions/6.0.0" --with-internal-glib --build=x86_64-apple-darwin13.0.0 --disable-dependency-tracking --cache-file=�[1m@�[0mexternal/bockbuild/builds/pkg-config-darwin-64.cache ac_cv_func_basename_r=no ac_cv_func_clock_getres=no ac_cv_func_clock_gettime=no ac_cv_func_clock_settime=no ac_cv_func_dirname_r=no ac_cv_func_getentropy=no ac_cv_func_mkostemp=no ac_cv_func_mkostemps=no ac_cv_func_timingsafe_bcmp=no
07:24:56  	@	make -j4
07:25:01  	@	make install DESTDIR=�[1m@�[0mexternal/bockbuild/scratch/pkg-config-x64.install
07:25:01  
07:25:01  darwin->postprocess:
07:25:01  	stage_textfiles: 0 items
07:25:01  	stage_binaries: 1 items
07:25:02  lipo pkg-config
07:25:02  
07:25:02  pkg-config->deploy_package:
07:25:02  	13 files, 2.00MB

in 6.0.0.223 https://jenkins.mono-project.com/view/Releases/job/build-package-osx-mono/job/2019-02/199/

@alexischr

This comment has been minimized.

Copy link
Member Author

@alexischr alexischr commented May 20, 2019

Yeah it seems to be an intermittent bug, that package is fine and locally I could not reproduce. @akoeplinger do you have a link to a recent broken package?

@alexischr alexischr reopened this May 29, 2019
@jaykrell

This comment has been minimized.

Copy link
Collaborator

@jaykrell jaykrell commented May 29, 2019

Just guessing, should:

--build=x86_64-apple-darwin13.0.0

be --host or --target?

Usually --host is what you care about.
--build is the current machine and fairly irrelevant, except as a default for the others.
--target is for only compilers, i.e. that output code.
--host is for non-compilers and compilers.

http://jingfenghanmax.blogspot.com/2010/09/configure-with-host-target-and-build.html

@alexischr

This comment has been minimized.

Copy link
Member Author

@alexischr alexischr commented May 30, 2019

@jaykrell I think tha should be fine to test (adding --target but not removing --build for sanity's sake) - however the issue confusingly does not show up in my attempts at building the package, either on my own workstation or one of our CI machines:

file /Library/Frameworks/Mono.framework/Versions/Current/bin/pkg-config64
/Library/Frameworks/Mono.framework/Versions/Current/bin/pkg-config64: Mach-O executable i386
file ~/mono/external/bockbuild/stage/bin/pkg-config64
/Users/alexischr/mono/external/bockbuild/stage/bin/pkg-config64: Mach-O 64-bit executable x86_64

But as you said, it's a default for the others, so I am not sure it will make a difference.

@jaykrell

This comment has been minimized.

Copy link
Collaborator

@jaykrell jaykrell commented May 30, 2019

Also, autoconf version maybe a factor? Can we capture/compare generated/copied configure/config.sub and config.log?

Is there a CI or Jenkins lane that built the bad one, with logs?

@jaykrell

This comment has been minimized.

Copy link
Collaborator

@jaykrell jaykrell commented May 30, 2019

Wild guess? Old cache file???

diff --git a/bockbuild/darwinprofile.py b/bockbuild/darwinprofile.py
index 4363991..051445a 100644
--- a/bockbuild/darwinprofile.py
+++ b/bockbuild/darwinprofile.py
@@ -150,8 +150,8 @@ class DarwinProfile (UnixProfile):
         configure_cache =  '%s/%s-%s.cache' % (self.bockbuild.build_root, package.name, arch)
         package.aux_files.append (configure_cache)

-        package.local_configure_flags.extend(
-            ['--cache-file=%s' % configure_cache])
+        #package.local_configure_flags.extend(
+        #    ['--cache-file=%s' % configure_cache])
alexischr added a commit to mono/bockbuild that referenced this issue Jun 10, 2019
… 32-bit package regardless of arch configuration

Mono issue: mono/mono#13804

The hypothesis is that at reconfigure using the configure cache file, the lack of explicit setting of the host somehow breaks the build process.
alexischr added a commit to alexischr/mono that referenced this issue Jun 10, 2019
… 32-bit package regardless of arch configuration

Mono issue: mono#13804

The hypothesis is that at reconfigure using the configure cache file, the lack of explicit setting of the host somehow breaks the build process.
@jaykrell

This comment has been minimized.

Copy link
Collaborator

@jaykrell jaykrell commented Jun 10, 2019

If adding —host fixes it, consider removing —build then.
Letting —build be detected should be ok.
It shouldn’t affect eventual output, when —host is explicit.
Right?

akoeplinger added a commit to mono/bockbuild that referenced this issue Jul 24, 2019
…built as a… (#112)

* Another speculative fix for 'pkg-config' sometimes getting built as a 32-bit package regardless of arch configuration

Mono issue: mono/mono#13804

The hypothesis is that at reconfigure using the configure cache file, the lack of explicit setting of the host somehow breaks the build process.

* Don't eagerly delete logs

Delete logs of built packages only if they are about to get rebuilt

Backport of #98.
akoeplinger added a commit to mono/bockbuild that referenced this issue Jul 24, 2019
…built as a… (#113)

* Another speculative fix for 'pkg-config' sometimes getting built as a 32-bit package regardless of arch configuration

Mono issue: mono/mono#13804

The hypothesis is that at reconfigure using the configure cache file, the lack of explicit setting of the host somehow breaks the build process.

* Don't eagerly delete logs

Delete logs of built packages only if they are about to get rebuilt

Backport of #98.
akoeplinger added a commit that referenced this issue Jul 24, 2019
akoeplinger added a commit that referenced this issue Jul 24, 2019
@akoeplinger

This comment has been minimized.

Copy link
Member

@akoeplinger akoeplinger commented Jul 24, 2019

I backported the PR to 2019-02 and up, let's hope it does really fix the issue :)

@lambdageek regarding your second point, that shouldn't matter since we're concatenating the array elements so the end result is the same.

lewurm added a commit to lewurm/xamarin-macios that referenced this issue Jul 26, 2019
New commits in mono/mono:

* mono/mono@761220d [sdks] Report xcode11 builds as a separate GitHub status
* mono/mono@f06da77 [sdks] Add special build job for building with Xcode 11
* mono/mono@c6949d7 Bump bockbuild to get pkg-config fix for mono/mono#13804
* mono/mono@0cd93e9 [sre] Make creation of dynamic method synchronized (#15814)
* mono/mono@d8286ce Bump mono/corefx@eb5ba09 (#15813)
* mono/mono@4b7a92e [runtime] Fix the error message for exceptions created from MonoError.
* mono/mono@31bf75e [2019-06] Fix ambiguous time at the end of DST period on Windo… (#15785)
* mono/mono@4f29e31 Add more tests to tarball with EXTRA_DIST
* mono/mono@afcf28a [2019-06][ios] Switch to python offset tool (#15756)
* mono/mono@5bd33de [ci] Clean working directory after build in package and sdks build
* mono/mono@a375e6a [packaging] Don't use dl.xamarin.com for gtksharp package anymore
* mono/mono@4db3b96 [sdks] Calculate sha1 of sdks archive and upload it
* mono/mono@ad226f5 [ci] Don't require master node for reportGitHubStatus method
* mono/mono@aef3c4c [sdks] Add linker sources to Android linux archive
* mono/mono@a2bcc63 [2019-06] [sdks] Add third party notices to sdks archives (#15755)
* mono/mono@c636636 Align mcs's tuple deconstruction behavior with csc
* mono/mono@1c919ea [2019-06] Print two bytes at a time in mono_pe_file_map conversion error (#15726)
* mono/mono@4b15f9b [mini] Runtime check if boxing is needed for DIM constrained calls (#15730)
* mono/mono@17520a8 Bump msbuild to msbuild:mono-2019-06 to stop disabling compiler server
* mono/mono@048d95d Add com.apple.security.cs.allow-unsigned-executable-memory exception to hardened runtime entitlements (#15725)
* mono/mono@c63edc3 [2019-06] [bcl] Sync recent NS2.1 changes (#15719)
* mono/mono@74e1f4c Revert "[ci] Remove separate utils.groovy file" and run step on master instead
* mono/mono@a79115d [2019-06] [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools… (#15718)
* mono/mono@d410e07 [ci] Remove separate utils.groovy file
* mono/mono@058d207 [ci] Wait for signing outside of a node() block
* mono/mono@a5b7755 [2019-06] [System] Keep events from firing from a disposed 'De… (#15706)
* mono/mono@ce68c20 [ci] Bump timeout for .pkg signing job

Diff: https://github.com/mono/mono/compare/edc9e70e04a2e5a7c760044db980c99496ad0003..761220d914e28b3ebd67e5642cf8c200110f62ec
mandel-macaque added a commit to xamarin/xamarin-macios that referenced this issue Jul 26, 2019
New commits in mono/mono:

* mono/mono@761220d [sdks] Report xcode11 builds as a separate GitHub status
* mono/mono@f06da77 [sdks] Add special build job for building with Xcode 11
* mono/mono@c6949d7 Bump bockbuild to get pkg-config fix for mono/mono#13804
* mono/mono@0cd93e9 [sre] Make creation of dynamic method synchronized (#15814)
* mono/mono@d8286ce Bump mono/corefx@eb5ba09 (#15813)
* mono/mono@4b7a92e [runtime] Fix the error message for exceptions created from MonoError.
* mono/mono@31bf75e [2019-06] Fix ambiguous time at the end of DST period on Windo… (#15785)
* mono/mono@4f29e31 Add more tests to tarball with EXTRA_DIST
* mono/mono@afcf28a [2019-06][ios] Switch to python offset tool (#15756)
* mono/mono@5bd33de [ci] Clean working directory after build in package and sdks build
* mono/mono@a375e6a [packaging] Don't use dl.xamarin.com for gtksharp package anymore
* mono/mono@4db3b96 [sdks] Calculate sha1 of sdks archive and upload it
* mono/mono@ad226f5 [ci] Don't require master node for reportGitHubStatus method
* mono/mono@aef3c4c [sdks] Add linker sources to Android linux archive
* mono/mono@a2bcc63 [2019-06] [sdks] Add third party notices to sdks archives (#15755)
* mono/mono@c636636 Align mcs's tuple deconstruction behavior with csc
* mono/mono@1c919ea [2019-06] Print two bytes at a time in mono_pe_file_map conversion error (#15726)
* mono/mono@4b15f9b [mini] Runtime check if boxing is needed for DIM constrained calls (#15730)
* mono/mono@17520a8 Bump msbuild to msbuild:mono-2019-06 to stop disabling compiler server
* mono/mono@048d95d Add com.apple.security.cs.allow-unsigned-executable-memory exception to hardened runtime entitlements (#15725)
* mono/mono@c63edc3 [2019-06] [bcl] Sync recent NS2.1 changes (#15719)
* mono/mono@74e1f4c Revert "[ci] Remove separate utils.groovy file" and run step on master instead
* mono/mono@a79115d [2019-06] [bcl] Add Mono.Cecil.Mdb assembly to monodroid_tools… (#15718)
* mono/mono@d410e07 [ci] Remove separate utils.groovy file
* mono/mono@058d207 [ci] Wait for signing outside of a node() block
* mono/mono@a5b7755 [2019-06] [System] Keep events from firing from a disposed 'De… (#15706)
* mono/mono@ce68c20 [ci] Bump timeout for .pkg signing job

Diff: https://github.com/mono/mono/compare/edc9e70e04a2e5a7c760044db980c99496ad0003..761220d914e28b3ebd67e5642cf8c200110f62ec
@akoeplinger

This comment has been minimized.

Copy link
Member

@akoeplinger akoeplinger commented Jul 31, 2019

I haven't seen this anymore in the latest 2019-06 packages so let's hope it's fixed for good now :)

@mrward

This comment has been minimized.

Copy link
Member

@mrward mrward commented Aug 15, 2019

There still seems to be a problem with the 2019-06 branch. I built a Xamarin.Mac project with VS Mac using Mono 6.4.0.165 and got a warning about pkg-config64 not being optimised for the Mac.

Installing the latest Mono 6.4 from the 2019-06 branch available today, removed the pkg-config64 file to ensure it was being replaced, and it seems that pkg-config64 is 32-bit.

file /Library/Frameworks/Mono.framework/Commands/pkg-config64
/Library/Frameworks/Mono.framework/Commands/pkg-config64: Mach-O executable i386

mono --version
Mono JIT compiler version 6.4.0.169 (2019-06/4a855304725 Wed Aug 14 14:40:14 EDT 2019)
@rolfbjarne

This comment has been minimized.

Copy link
Member

@rolfbjarne rolfbjarne commented Aug 16, 2019

I can confirm, this still happens with the tip of 2019-06, and customers are running into it to (https://dev.azure.com/devdiv/DevDiv/_workitems/edit/961683).

@rolfbjarne rolfbjarne reopened this Aug 16, 2019
akoeplinger added a commit to mono/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
akoeplinger added a commit to mono/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
monojenkins added a commit to monojenkins/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
monojenkins added a commit to monojenkins/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
monojenkins added a commit to monojenkins/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
akoeplinger added a commit to mono/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py (#117)

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
akoeplinger added a commit to mono/bockbuild that referenced this issue Aug 29, 2019
…uartz-engine.py (#118)

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
akoeplinger added a commit to mono/bockbuild that referenced this issue Aug 30, 2019
…uartz-engine.py (#119)

We hit troubles (mono/mono#13804) which are apparently related to lipo'ing so move the three packages that use it away from it.

- pkg-config doesn't build as universal since the internal glib doesn't build because of 32/64bit configure mismatches (we'd need to do similar patches to what we do in glib.py) so just make it 64bit only.

- libtiff also runs into the 32/64bit configure mismatch but we can fix it with a simple patch.
  I tested it works by running the repro from https://xamarin.github.io/bugzilla-archives/39/39865/bug.html .

- gtk-quartz-engine.py is no longer used by MD/VSMac according to the team so remove it completely.
@akoeplinger

This comment has been minimized.

Copy link
Member

@akoeplinger akoeplinger commented Aug 30, 2019

I switched pkg-config to be 64bit only with mono/bockbuild#116 which should fix this once and for all.

It's included in these Mono versions:

  • 2019-02 / 20aebce - Mono 6.0.0.333
  • 2019-06 / 7ef99c0 - Mono 6.4.0.186
  • 2019-08 / 23cc371 - Mono 6.6.0.48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.