Skip to content

/ servo

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Intermittent mac android linking error ("fatal error: out of file descriptors and couldn't close any") #23409

Closed
jdm opened this issue May 16, 2019 · 14 comments
Closed

Intermittent mac android linking error ("fatal error: out of file descriptors and couldn't close any") #23409

jdm opened this issue May 16, 2019 · 14 comments
Labels
A-build I-intermittent P-mac

Comments

@jdm
Copy link
Member

@jdm jdm commented May 16, 2019 

  = note: /Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: fatal error: out of file descriptors and couldn't close any
          collect2: error: ld returned 1 exit status
          toolchain: /Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin
          libs dir: /Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a
          sysroot: /Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/platforms/android-21/arch-arm
          targetdir: /Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a
          

error: aborting due to previous error

The following warnings were emitted during compilation:

warning: clang: warning: argument unused during compilation: '-L/Users/worker/tasks/task_1558012770/repo/android-toolchains/ndk/platforms/android-21/arch-arm/usr/lib' [-Wunused-command-line-argument]

error: Could not compile `simpleservo_jniapi`.
@SimonSapin
Copy link
Member

@SimonSapin SimonSapin commented May 16, 2019

Does this happen on some workers and not at all on some others?
@jdm
Copy link
Member Author

@jdm jdm commented May 16, 2019

Hard to say so far.
@jdm
Copy link
Member Author

@jdm jdm commented May 16, 2019 •
edited

I've checked and the file handles limit on the workers match my local machine - soft limit of 256, and hard limit of unlimited. Used https://superuser.com/questions/1369590/launchctl-limit-maxfiles-vs-sysctl-kern-maxfilesperproc as a reference.
@jdm
Copy link
Member Author

@jdm jdm commented May 16, 2019

However, the buildbot mac builders have sysctl kern.maxfiles reporting kern.maxfiles: 65536, while the TC builders that were reimaged report kern.maxfiles: 12288.
@jdm
Copy link
Member Author

@jdm jdm commented May 16, 2019

Similarly, buildbot builders have sysctl kern.maxfilesperproc report kern.maxfilesperproc: 30000 and TC builders report kern.maxfilesperproc: 10240.
@jdm
Copy link
Member Author

@jdm jdm commented May 19, 2019

More specific, mac3, mac5, and mac7 have the lower maxfilesperproc and maxfiles limits. The remaining machines have 24576 and 49152 respectively.
@jdm
Copy link
Member Author

@jdm jdm commented May 19, 2019

I ran the following:

godot:~ jdm$ for i in 3 5 7; do echo -n "$i "; ssh root@servo-tc-mac$i.servo.org 'sysctl -w kern.maxfiles=49152; sysctl -w kern.maxfilesperproc=24576'; done
3 kern.maxfiles: 12288 -> 49152
kern.maxfilesperproc: 10240 -> 24576
5 kern.maxfiles: 12288 -> 49152
kern.maxfilesperproc: 10240 -> 24576
7 kern.maxfiles: 12288 -> 49152
kern.maxfilesperproc: 10240 -> 24576
@jdm
Copy link
Member Author

@jdm jdm commented May 19, 2019

Hit this again on servo-tc-mac7 since running those commands :(
@jdm
Copy link
Member Author

@jdm jdm commented May 20, 2019

Hit this on servo-tc-mac4 for the first time, sadly.
@jdm
Copy link
Member Author

@jdm jdm commented May 21, 2019

Observed on servo-tc-mac9 as well.
@jdm jdm mentioned this issue May 21, 2019
Merged
bors-servo added a commit that referenced this issue May 21, 2019
@bors-servo
Auto merge of #23435 - servo:jdm-patch-51, r=Manishearth
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
f9e5e7f 
Disable sccache on macOS

This is an attempt to see if sccache is responsible for #23409 and #23428.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23435)
<!-- Reviewable:end -->
@jdm
Copy link
Member Author

@jdm jdm commented May 22, 2019

This was observed on servo-tc-mac4 after #23435. Oh well.
@jdm
Copy link
Member Author

@jdm jdm commented May 22, 2019

I have run launchctl unload -w /system/library/launchdaemons/com.apple.metadata.mds.plist on every mac builder to see if this makes a difference.
@jdm jdm mentioned this issue May 22, 2019
Merged
@jdm
Copy link
Member Author

@jdm jdm commented May 22, 2019

No apparent difference.
bors-servo added a commit that referenced this issue May 22, 2019
@bors-servo
Auto merge of #23442 - jdm:no-glutin-up, r=jdm
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
a1fba8d 
Revert glutin upgrade.

#23409 is permanently affecting all macOS android builds starting with this PR.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23442)
<!-- Reviewable:end -->
bors-servo added a commit that referenced this issue May 22, 2019
@bors-servo
Auto merge of #23442 - jdm:no-glutin-up, r=jdm
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
f438f68 
Revert glutin upgrade.

#23409 is permanently affecting all macOS android builds starting with this PR.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23442)
<!-- Reviewable:end -->
bors-servo added a commit that referenced this issue May 23, 2019
@bors-servo
Auto merge of #23442 - jdm:no-glutin-up, r=jdm
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
e0c95ed 
Revert glutin upgrade.

#23409 is permanently affecting all macOS android builds starting with this PR.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23442)
<!-- Reviewable:end -->
@CYBAI CYBAI mentioned this issue May 23, 2019
Merged
4 of 5 tasks complete
@jdm
Copy link
Member Author

@jdm jdm commented Jun 3, 2019

Haven't seen this in a while.
@jdm jdm closed this Jun 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-build I-intermittent P-mac
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@jdm @SimonSapin
You can’t perform that action at this time.