Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-1908] Foundation Thread exhaustion #4560

Closed
swift-ci opened this issue Jun 26, 2016 · 8 comments
Closed

[SR-1908] Foundation Thread exhaustion #4560

swift-ci opened this issue Jun 26, 2016 · 8 comments

Comments

@swift-ci
Copy link
Contributor

swift-ci commented Jun 26, 2016

Previous ID SR-1908
Radar None
Original Reporter joeiachievedit (JIRA User)
Type Bug
Status Closed
Resolution Done
Environment

Thread exhaustion observed on a Raspberry Pi 3 running Xenial Xerus. If anyone needs assistance/login to a Pi3 to run and observe please let me (joe@iachieved.it) know.

Additional Detail from JIRA
Votes 0
Component/s Foundation
Labels Bug, Linux
Assignee joeiachievedit (JIRA)
Priority Medium

md5: 656708644b9554713d9e2484ece916f0

Issue Description:

I discovered this while working with a Raspberry Pi 3, as the current implementation of Thread exhausts much faster on it than compared to, say, an x86 server with 16G of RAM.

Consider the following:

import Foundation
import Glibc

var counter = 0
while true {
  sleep(2)
  counter = counter + 1
  let t = NSThread(){
    print("STARTED:\(counter)")
    sleep(1)
    print("EXIT:\(counter)")
    NSThread.exit()
  }
  print("START:\(counter)")
  t.start()
}

The sleep(2) is designed to space thread creation out to such that only one thread is running at a time. On a Raspberry Pi 3 this code makes it this far:

START:231
STARTED:231
EXIT:231
START:232
STARTED:232
EXIT:232
START:233
STARTED:233
EXIT:233

After this subsequent calls to thread.start() return silently given that Thread.start() has no return value:

START:234
START:235
START:236
START:237
START:238
START:239
START:240
START:241
START:242
START:243

An x86 server equivalent suffers the same fate but in hours rather than minutes.

@phausler
Copy link
Member

phausler commented Jun 27, 2016

Interesting, Thread.exit() just calls to pthread_exit; I wonder if there is some other sort of underlying failure there.

@phausler
Copy link
Member

phausler commented Jun 27, 2016

I don't have a pi handy to test this out; but the one differential between the Darwin implementation of the objective-c version and the swift-corelibs-foundation version is that the thread attributes for the objective-c version has a few initializers

    pthread_attr_init(&ivars->attr);
    pthread_attr_setscope(&ivars->attr, PTHREAD_SCOPE_SYSTEM);
    pthread_attr_setdetachstate(&ivars->attr, PTHREAD_CREATE_DETACHED);

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 27, 2016

Comment by Joe (JIRA)

@phausler: I believe I can patch these initializers in and kick the tires. I'll provide the patch that I came up with and report back.

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 28, 2016

Comment by Joe (JIRA)

@phausler: I applied the general patch into NSThread.swift and am running it now. Up to 300 threads created and exited on the Pi 3. Will let it run up a bit higher and then provide a PR. For reference using your code as a hint, Linux creates threads as JOINABLE by default, thus with no one joining the app was leaking thread resources. Changing to DETACHED resolved that. I don't believe SCOPE_SYSTEM has any effect as this apparently is the only scope actually supported on Linux.

References:
http://man7.org/linux/man-pages/man3/pthread_attr_setscope.3.html
https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/Multithreading/CreatingThreads/CreatingThreads.html#//apple_ref/doc/uid/10000057i-CH15-SW3
"Setting the Detached State of a Thread
Most high-level thread technologies create detached threads by default. In most cases, detached threads are preferred because they allow the system to free up the thread’s data structures immediately upon completion of the thread. Detached threads also do not require explicit interactions with your program. The means of retrieving results from the thread is left to your discretion. By comparison, the system does not reclaim the resources for joinable threads until another thread explicitly joins with that thread, a process which may block the thread that performs the join.

You can think of joinable threads as akin to child threads. Although they still run as independent threads, a joinable thread must be joined by another thread before its resources can be reclaimed by the system. Joinable threads also provide an explicit way to pass data from an exiting thread to another thread. Just before it exits, a joinable thread can pass a data pointer or other return value to the pthread_exit function. Another thread can then claim this data by calling the pthread_join function."

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 28, 2016

Comment by Joe (JIRA)

Fix looks good. Shortened delay of creation/destruction of threads:

import Foundation
import Glibc

var counter = 0
while true {
  usleep(100)
  counter = counter + 1
  let t = Thread(){
    print("STARTED:\(counter)")
    usleep(50)
    print("EXIT:\(counter)")
    Thread.exit()
  }
  print("START:\(counter)")
  t.start()
}

This is running steady state on a Pi 3 with over 1 million threads created and destroyed. That was compared to ~250 without the patch.

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 28, 2016

Comment by Joe (JIRA)

#425 PR submitted.

Pi 3 at 1.7M thread creates/destroys.

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 28, 2016

Comment by Joe (JIRA)

Merge complete, will rebuild, verify, and close.

@swift-ci
Copy link
Contributor Author

swift-ci commented Jun 28, 2016

Comment by Joe (JIRA)

Verified fixed against e74f745 Closing.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 6, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants