support foreign threads

[JeffBezanson]

Rebased #45447 and moved to this repo.

Fixes #17573

[IanButterworth]

It would be good to fix (remove) the compat note in the docstring for Profile.init in the first commit here, as that commit will I assume be backported to 1.8

[vtjnash]

This assertion seems odd:

      From worker 2:	Assertion failed: gc_num.interval == default_collect_interval, file /cygdrive/c/buildbot/worker/package_win32/build/src/gc.c, line 3582
      From worker 2:	
      From worker 2:	[50132] signal (22): SIGABRT
      From worker 2:	in expression starting at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\ccall.jl:1059
      From worker 2:	Allocations: 638253607 (Pool: 638037914; Big: 215693); GC: 2048
Worker 2 terminated.
ccall                                            (2) |         failed at 2022-09-02T20:53:24.341

[gbaraldi]

There is an rr trace for that in the rr testset.

[JeffBezanson]

This assertion seems odd:

Yeah I don't see why we would need that. Adding threads after startup makes that assertion straightforwardly untrue.

[JeffBezanson]

I don't get the assertion error running the ccall test, but instead I get

  LoadError: TypeError: in ccall argument 1, expected Int32, got a value of type Int32
  Stacktrace:
    [1] top-level scope
      @ ~/src/julia/test/ccall.jl:1098

and have also seen memory corruption during bootstrap, so something must be going wrong here.

[JeffBezanson]

OK the Int32 thing is just due to the IR verifier not handling cglobal, but with GC_VERIFY I get

    JULIA usr/lib/julia/corecompiler.ji

[551] signal (11): Segmentation fault
in expression starting at compiler/bootstrap.jl:10
gc_verify_tags_page at /home/jeffbezanson/src/julia/src/gc-debug.c:352
gc_verify_tags_pagetable0 at /home/jeffbezanson/src/julia/src/gc-debug.c:366
gc_verify_tags_pagetable1 at /home/jeffbezanson/src/julia/src/gc-debug.c:380
gc_verify_tags_pagetable at /home/jeffbezanson/src/julia/src/gc-debug.c:394
gc_verify_tags at /home/jeffbezanson/src/julia/src/gc-debug.c:433
_jl_gc_collect at /home/jeffbezanson/src/julia/src/gc.c:3344
ijl_gc_collect at /home/jeffbezanson/src/julia/src/gc.c:3499

[JeffBezanson]

Ok, never mind, that has been happening for a long time at least since 1.7...
Looks like MEMFENCE is broken.

[oscardssmith]

Ready to merge?

[JeffBezanson]

@vtjnash Please re-review jl_atexit_hook --- I'm not sure what the scope of the gc unsafe region should be. It was a bit oddly nested before.

[DilumAluthge]

Is there any chance that this PR broke i686-linux-gnu on master?

[ancapdev]

From NEWS

Threads started outside the Julia runtime (e.g. from C or Java) can now become able to call into Julia code by calling jl_adopt_thread. This is done automatically when entering Julia code via cfunction or a @ccallable entry point

Presumably this means there's now extra overhead to entering a cfunction to perform that check, and code built on that or wrappers like FunctionWrappers.jl will suffer a performance regression in 1.9?

[vtjnash]

The overhead to support this was added pre-1.0, the feature is just finally enabled now

[ancapdev]

Interesting, didn't know there was already a dead check and branch in every cfunction, thanks for clarifying.

Separate topic I guess, but that and other reasons would make it compelling to add native typed functions, and a way to bind type erased closures to such function objects with no other overhead than an indirect call and with full support of all types (i.e., not limited to what passes through the Julia C ABI).