Add create_library function

.gitignore

@@ -4,6 +4,7 @@
 .DS_Store
 /Manifest.toml
 /examples/MyApp/MyApp
+/examples/*Compiled
 /dev/
 *.so
 *.dll

README.md

@@ -4,12 +4,14 @@
 [![Codecov](https://codecov.io/gh/JuliaLang/PackageCompiler.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaLang/PackageCompiler.jl)
 [![][docs-stable-img]][docs-stable-url]
 
-PackageCompiler is a Julia package with two main purposes:
+PackageCompiler is a Julia package with three main purposes:
 
   1. Creating custom sysimages for reduced latency when working locally with packages that has a high startup time.
 
   2. Creating "apps" which are a bundle of files including an executable that can be sent and run on other machines without Julia being installed on that machine.
 
+  3. Creating a relocatable C library bundle from of Julia code.
+
 For installation and usage instructions, see the [documentation][docs-stable-url].
 
 [docs-stable-img]: https://img.shields.io/badge/docs-stable-blue.svg

docs/src/apps.md

@@ -12,7 +12,7 @@ original Julia source code for apps since everything gets baked into the
 sysimage.
 
 
-## Relocatability
+## Relocatability(@id relocatability)
 
 Since we want to send the app to other machines the app we create must be
 "relocatable".  With an app being relocatable we mean it does not rely on

docs/src/index.md

@@ -6,7 +6,7 @@ take some time. All subsequent calls within that same session use this fast comp
 but if you restart Julia you lose all the compiled work. PackageCompiler allows you to do this
 work up front — further ahead of time — and store the results for a lower latency startup.
 
-There are two main workflows:
+There are three main workflows:
 
 1. You can save loaded packages and compiled functions into a file (called a
     [sysimage](@ref sysimages)) that you pass to `julia` upon startup. Typically the
@@ -22,7 +22,12 @@ There are two main workflows:
     potentially cross-platfrom binary libraries) and Julia itself with a single executable
     as its entry point.
 
-The most challenging part in both cases is in determining _which_ methods need to be
+3. Alternatively, you can create a [C library](@ref libs).  In this case, your package should
+    define c-callable functions to be included in the sysimage.  As with apps, generating a
+    library bundles together Julia and all dependencies in a (hopefully) redistributable
+    directory structure that can be moved to other machines (of the same architecture).
+
+The most challenging part in all cases is in determining _which_ methods need to be
 compiled ahead of time. For example, your project will certainly require addition between
 integers (`+(::Int, ::Int)`; thankfully that's already compiled into Julia itself), but does
 it require addition of dates (like `+(::Date, ::Day)` or `+(::DateTime, ::Hour)`)? What

docs/src/libs.md

@@ -0,0 +1,120 @@
+# [Libraries](@id libraries)
+
+Creating a library with PackageCompiler involves creating a custom system image with a couple of
+additional features to facilitate linking and use by external (non-Julian) programs--it's already
+a dynamic library.
+
+As with app creation, we distribute all of the libraries necessary to run Julia.  In the end, we
+end up with a directory of libraries (`lib`, or `bin` on Windows), an `include` directory
+with C header files, and an `artifacts` directory (for any additional library artifacts needed
+by the Julia runtime).
+
+Of course, we also want the library to be relocatable, and all of the caveats
+in the [App relocatability](@ref relocatability) section still apply.
+
+## Creating a library
+
+As with apps, the source of a library is a package with a project and manifest file.
+The library is expected to provide C-callable functions for the functionality it
+is providing, defined using the `Base.@ccallable` macro:
+
+```julia
+Base.@ccallable function increment(count::Cint)::Cint
+    count += 1
+    println("Incremented count: $count")
+    return count
+end
+```
+
+All C-callable functions will be exported and made available to programs which link to the 
+library.
+
+A skeleton of a library to start working from can be found 
+[here](https://github.com/JuliaLang/PackageCompiler.jl/tree/master/examples/MyLib).
+
+A complete example which works on all supported 64-bit OS platforms can be found [here](https://github.com/simonbyrne/libcg).
+(32-bit is not yet working there).
+
+The library is compiled using the [`create_library`](@ref) function, which takes the path to the
+source code and the destination directory.  
+
+```
+~/PackageCompiler.jl/examples
+❯ julia -q --project
+
+julia> using PackageCompiler
+
+julia> create_library("MyLib", "MyLibCompiled";
+                      lib_name="libinc",
+                      precompile_execution_file="MyLib/build/generate_precompile.jl",
+                      precompile_statements_file="MyLib/build/additional_precompile.jl",
+                      header_files = ["MyLib/inc.h"])
+┌ Warning: it is not recommended to create an app without a preexisting manifest
+└ @ PackageCompiler ~/.julia/dev/PackageCompiler/src/PackageCompiler.jl:903
+[ Info: PackageCompiler: creating base system image (incremental=false)...
+[ Info: PackageCompiler: creating system image object file, this might take a while...
+[ Info: PackageCompiler: creating system image object file, this might take a while...
+
+julia> exit()
+
+~/PackageCompiler.jl/examples
+❯ ls -al MyLibCompiled/lib/libinc.*
+-rwxr-xr-x  1 kmsquire  staff  97241152 Jan 28 14:27 MyLibCompiled/lib/libinc.dylib
+
+~/PackageCompiler.jl/examples
+❯ ls MyLibCompiled/lib   # MyLibCompiled/bin on Windows
+julia/
+libinc.dylib
+libjulia.1.6.dylib
+libjulia.1.dylib
+libjulia.dylib
+...
+```
+
+(These will have an `.so` extension on Linux, and a `.dll` extension on Windows.  There may also
+be other files in the same directory, depending on your operating system and version of Julia.)
+
+In addition to most of the same keyword arguments as 
+[`create_app`](@ref), `create_libraray` has additional keyword arguments related to library
+naming, versioning, and including C header files in the output library bundle.  See the function
+documentation for details.
+
+Presumably you're creating the library to use some functionality that is available in Julia
+but not (easily) implementable in some other language, like C or C++.  To use this functionality
+from, e.g., C, you'll need to link against the library, and also make it accessible at run time
+(because it's a dynamic library, not a static one).
+
+Here you have different options depending on your operating system and needs.
+
+1. Install the libraries in a non-standard location, and update an appropriate environment
+   variable to point to the library location.
+   * On Linux and other unix-like OSes, run `export LD_LIBRARY_PATH=/path/to/lib:$LD_LIBRARY_PATH`
+   * On Mac, run `export DYLD_FALLBACK_LIBRARY_PATH=/path/to/lib:$DYLD_FALLBACK_LIBRARY_PATH`
+   * On Windows, include the library location in `PATH`. (* NOTE: not tested--does this work? *)
+
+2. (Linux/Unix/Mac) Install the library files in a standard library location. `/usr/local/`
+   is one possible location:
+   * Libraries would be installed in `/usr/local/lib`
+   * Include files would be installed in `/usr/local/include`
+   * Julia artifacts and any other depot components would be installed under `/usr/local/share/julia`.
+
+   NOTE: On Linux, installing under `/usr/local/lib` or another standard location requires 
+         that you run `ldconfig` as root after install.
+
+3. (Mac) Include the full library bundle in an application bundle, and set the `rpath`
+   on the application bundle to the relative path of the library from the executable.
+
+4. (Windows) Include all libraries in the same directory as an executable.
+
+In all cases, you also need to link to the library while building your executable.  For C/C++
+compilers, the link step looks something like this:
+
+```
+cc -o my_application my_application.o -L/path/to/my_library -lmylib
+```
+
+Note that on Unix-like operating systems (including Mac), your library must have a `lib` prefix
+(e.g., `libmylib.so` (linux/unix) or `libmylib.dylib` (Mac)).  `create_library()` ensures this.
+(On windows, the `lib` prefix is optional.)
+
+See [here](https://github.com/simonbyrne/libcg) for a more complete example of how this might look.

docs/src/refs.md

@@ -4,5 +4,6 @@
 PackageCompiler.create_sysimage
 PackageCompiler.restore_default_sysimage
 PackageCompiler.create_app
+PackageCompiler.create_library
 PackageCompiler.audit_app
 ```

examples/MyLib/Makefile

@@ -0,0 +1,21 @@
+# Makefile
+
+JULIA ?= julia
+DLEXT := $(shell $(JULIA) --startup-file=no -e 'using Libdl; print(Libdl.dlext)')
+
+TARGET="../MyLibCompiled"
+
+MYLIB_INCLUDES = $(TARGET)/include/julia_init.h $(TARGET)/include/mylib.h
+MYLIB_PATH := $(TARGET)/lib/libmylib.$(DLEXT)
+
+$(MYLIB_PATH) $(MYLIB_INCLUDES): build/build.jl src/MyLib.jl
+	julia --startup-file=no --project=. -e 'using Pkg; Pkg.instantiate()'
+	# Remove the following line when `create_library()` is merged upstream
+	$(JULIA) --startup-file=no --project=build -e 'import Pkg; Pkg.add(url="https://github.com/kmsquire/PackageCompiler.jl.git", rev="kms/create_library")'
+	julia --startup-file=no --project=build -e 'using Pkg; Pkg.instantiate()'
+	julia --startup-file=no --project=build build/build.jl
+
+.PHONY: clean
+clean:
+	$(RM) *~ *.o *.$(DLEXT)
+	$(RM) -Rf $(TARGET)

examples/MyLib/Manifest.toml

@@ -0,0 +1,2 @@
+# This file is machine-generated - editing it directly is not advised
+

examples/MyLib/Project.toml

@@ -0,0 +1,4 @@
+name = "MyLib"
+uuid = "a1a1a4a2-71b6-406f-b13d-8701d2765a5f"
+authors = ["Kevin Squire <kevin.squire@gmail.com>"]
+version = "0.1.0"

examples/MyLib/build/Manifest.toml

@@ -0,0 +1,118 @@
+# This file is machine-generated - editing it directly is not advised
+
+[[ArgTools]]
+uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
+
+[[Artifacts]]
+uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
+
+[[Base64]]
+uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
+
+[[Dates]]
+deps = ["Printf"]
+uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
+
+[[Downloads]]
+deps = ["ArgTools", "LibCURL", "NetworkOptions"]
+uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
+
+[[InteractiveUtils]]
+deps = ["Markdown"]
+uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
+
+[[LibCURL]]
+deps = ["LibCURL_jll", "MozillaCACerts_jll"]
+uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
+
+[[LibCURL_jll]]
+deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
+uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
+
+[[LibGit2]]
+deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
+uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
+
+[[LibSSH2_jll]]
+deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
+uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
+
+[[Libdl]]
+uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
+
+[[Logging]]
+uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
+
+[[Markdown]]
+deps = ["Base64"]
+uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
+
+[[MbedTLS_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
+
+[[MozillaCACerts_jll]]
+uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
+
+[[NetworkOptions]]
+uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
+
+[[PackageCompiler]]
+deps = ["Libdl", "Pkg", "UUIDs"]
+git-tree-sha1 = "0d7a10707b52d1db292e6990926f1a71c40ac9c7"
+repo-rev = "kms/create_library"
+repo-url = "https://github.com/kmsquire/PackageCompiler.jl.git"
+uuid = "9b87118b-4619-50d2-8e1e-99f35a4d4d9d"
+version = "1.2.5"
+
+[[Pkg]]
+deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
+uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
+
+[[Printf]]
+deps = ["Unicode"]
+uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
+
+[[REPL]]
+deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
+uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
+
+[[Random]]
+deps = ["Serialization"]
+uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
+
+[[SHA]]
+uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
+
+[[Serialization]]
+uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
+
+[[Sockets]]
+uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
+
+[[TOML]]
+deps = ["Dates"]
+uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
+
+[[Tar]]
+deps = ["ArgTools", "SHA"]
+uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
+
+[[UUIDs]]
+deps = ["Random", "SHA"]
+uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
+
+[[Unicode]]
+uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
+
+[[Zlib_jll]]
+deps = ["Libdl"]
+uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
+
+[[nghttp2_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
+
+[[p7zip_jll]]
+deps = ["Artifacts", "Libdl"]
+uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

examples/MyLib/build/Project.toml

@@ -0,0 +1,3 @@
+[deps]
+Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
+PackageCompiler = "9b87118b-4619-50d2-8e1e-99f35a4d4d9d"

examples/MyLib/build/additional_precompile.jl

@@ -0,0 +1 @@
+precompile(Tuple{typeof(MyLib.increment64), Clong})

examples/MyLib/build/build.jl

@@ -0,0 +1,16 @@
+using PackageCompiler
+
+target_dir = get(ENV, "OUTDIR", "$(@__DIR__)/../../MyLibCompiled")
+target_dir = replace(target_dir, "\\"=>"/")       # Change Windows paths to use "/"
+
+package_dir =
+
+println("Creating library in $target_dir")
+PackageCompiler.create_library(".", target_dir;
+                                lib_name="mylib",
+                                precompile_execution_file=["$(@__DIR__)/generate_precompile.jl"],
+                                precompile_statements_file=["$(@__DIR__)/additional_precompile.jl"],
+                                incremental=false,
+                                filter_stdlibs=true,
+                                header_files = ["$(@__DIR__)/mylib.h"],
+                            )

examples/MyLib/build/generate_precompile.jl

@@ -0,0 +1,10 @@
+using MyLib
+
+function count_to_ten()
+    count = zero(Int32)
+    while count < 10
+        count = increment32(count)
+    end
+end
+
+count_to_ten()

examples/MyLib/build/mylib.h

@@ -0,0 +1,2 @@
+int increment32(int);
+long increment64(long);

examples/MyLib/src/MyLib.jl

@@ -0,0 +1,22 @@
+module MyLib
+
+export increment32, increment64
+
+function increment(count)
+    count += 1
+    return count
+end
+
+Base.@ccallable function increment32(count::Cint)::Cint
+    count = increment(count)
+    println("Incremented count: $count (Cint)")
+    return count
+end
+
+Base.@ccallable function increment64(count::Clong)::Clong
+    count = increment(count)
+    println("Incremented count: $count (Clong)")
+    return count
+end
+
+end