Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize LLVM image generation #47797

Merged
merged 34 commits into from
Mar 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
615b142
Simplify multiversioning
pchintalapudi Jan 3, 2023
27808e1
Refactor aotcompile
pchintalapudi Jan 3, 2023
4524987
Timing print statements
pchintalapudi Jan 3, 2023
094269c
Move image init to processor.cpp
pchintalapudi Jan 5, 2023
2c7375c
Annotate information before running optimization
pchintalapudi Jan 6, 2023
6ab1862
Table-based dlsym
pchintalapudi Jan 6, 2023
798ee22
Allow loader to deal with multiple shards
pchintalapudi Jan 6, 2023
3915101
Multithreaded image builder
pchintalapudi Jan 7, 2023
4ad943d
Don't try to extract indexes during partitioning
pchintalapudi Jan 7, 2023
d717fa7
Fix whitespace
pchintalapudi Jan 7, 2023
fe0600d
Fix warnings
pchintalapudi Jan 7, 2023
bdf65f4
Set reloc slot to be external linkage
pchintalapudi Jan 7, 2023
4fc5bed
Formalize printing more, correct module weight estimation with multiv…
pchintalapudi Jan 10, 2023
0c68e4a
Alter naming, sort partitions
pchintalapudi Jan 26, 2023
f9da0e2
Fix whitespace
pchintalapudi Jan 26, 2023
9a72be6
Avoid unused function warning
pchintalapudi Jan 27, 2023
1f07ea5
Check relocations for generic target as well
pchintalapudi Jan 27, 2023
83b1967
Debug macos linker
pchintalapudi Jan 27, 2023
c98ff30
Respect JULIA_CPU_THREADS
pchintalapudi Jan 27, 2023
8cf48f2
Don't inject CRT aliases on macos
pchintalapudi Jan 27, 2023
4e35f41
Clean up timers and prints, link to JULIA_IMAGE_TIMINGS
pchintalapudi Feb 1, 2023
a723211
Fix whitespace
pchintalapudi Feb 1, 2023
7cf839a
Don't leave aliases to extern global objects
pchintalapudi Feb 1, 2023
fa208d4
Break multiversioning's dependency on jl_get_llvm_clone_targets
pchintalapudi Feb 1, 2023
3dcd1a2
Add multiversioning annotation test
pchintalapudi Feb 1, 2023
b3d3ffb
Couple more tests for multiversioning
pchintalapudi Feb 2, 2023
e75e362
Inject CRT aliases with internal linkage within every shard
pchintalapudi Feb 2, 2023
65e6de2
Expand on the multiversioning tests
pchintalapudi Feb 3, 2023
5561223
Remove stray debug prints
pchintalapudi Feb 3, 2023
fef319c
Track gvar count
pchintalapudi Feb 13, 2023
acc54d9
Add more assertions
pchintalapudi Feb 17, 2023
27f1ccd
Move dbgs under LLVM_DEBUG
pchintalapudi Mar 6, 2023
6b8ec27
Add some documentation
pchintalapudi Mar 6, 2023
5108b40
Update documentation
pchintalapudi Mar 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/src/devdocs/sysimg.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ as many platforms as possible, so as to give vastly improved startup times. On
not ship with a precompiled system image file, one can be generated from the source files shipped
in Julia's `DATAROOTDIR/julia/base` folder.

Julia will by default generate its system image on half of the available system threads. This
may be controlled by the [`JULIA_IMAGE_THREADS`](@ref env-image-threads) environment variable.

This operation is useful for multiple reasons. A user may:

* Build a precompiled shared library system image on a platform that did not ship with one, thereby
Expand Down
15 changes: 14 additions & 1 deletion doc/src/manual/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ To use Visual Studio Code on Windows, set `$JULIA_EDITOR` to `code.cmd`.

## Parallelization

### `JULIA_CPU_THREADS`
### [`JULIA_CPU_THREADS`](@id env-cpu-threads)

Overrides the global variable [`Base.Sys.CPU_THREADS`](@ref), the number of
logical CPU cores available.
Expand Down Expand Up @@ -316,6 +316,19 @@ then spinning threads never sleep. Otherwise, `$JULIA_THREAD_SLEEP_THRESHOLD` is
interpreted as an unsigned 64-bit integer (`uint64_t`) and gives, in
nanoseconds, the amount of time after which spinning threads should sleep.

### [`JULIA_IMAGE_THREADS`](@id env-image-threads)

An unsigned 32-bit integer that sets the number of threads used by image
compilation in this Julia process. The value of this variable may be
ignored if the module is a small module. If left unspecified, the smaller
of the value of [`JULIA_CPU_THREADS`](@ref env-cpu-threads) or half the
number of logical CPU cores is used in its place.

### `JULIA_IMAGE_TIMINGS`

A boolean value that determines if detailed timing information is printed during
during image compilation. Defaults to 0.

### `JULIA_EXCLUSIVE`

If set to anything besides `0`, then Julia's thread policy is consistent with
Expand Down
2 changes: 1 addition & 1 deletion src/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ $(BUILDDIR)/julia_flisp.boot: $(addprefix $(SRCDIR)/,jlfrontend.scm flisp/aliase

# additional dependency links
$(BUILDDIR)/codegen-stubs.o $(BUILDDIR)/codegen-stubs.dbg.obj: $(SRCDIR)/intrinsics.h
$(BUILDDIR)/aotcompile.o $(BUILDDIR)/aotcompile.dbg.obj: $(SRCDIR)/jitlayers.h $(SRCDIR)/llvm-codegen-shared.h
$(BUILDDIR)/aotcompile.o $(BUILDDIR)/aotcompile.dbg.obj: $(SRCDIR)/jitlayers.h $(SRCDIR)/llvm-codegen-shared.h $(SRCDIR)/processor.h
$(BUILDDIR)/ast.o $(BUILDDIR)/ast.dbg.obj: $(BUILDDIR)/julia_flisp.boot.inc $(SRCDIR)/flisp/*.h
$(BUILDDIR)/builtins.o $(BUILDDIR)/builtins.dbg.obj: $(SRCDIR)/iddict.c $(SRCDIR)/builtin_proto.h
$(BUILDDIR)/codegen.o $(BUILDDIR)/codegen.dbg.obj: $(addprefix $(SRCDIR)/,\
Expand Down
Loading