From a88697c0eded20076ba2f2cba3845718e4f003e9 Mon Sep 17 00:00:00 2001 From: Chris Nuernberger Date: Sat, 25 May 2019 15:14:40 -0600 Subject: [PATCH] Edited without gettting latest. --- README.md | 64 ++++++++++++++++++++++++++- java/libpython_clj/jna/JVMBridge.java | 5 ++- project.clj | 3 +- 3 files changed, 67 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index ac2077a..3a09308 100644 --- a/README.md +++ b/README.md @@ -10,18 +10,78 @@ advantages over a [10,000 line C binding layer](https://github.com/ninia/jep/tre ...if it works. +## Key Design Points + + +### Code Organization + + +There are 3 rough sections of code: +1. A JNA layer which is a 1-1 mapping most of the C API with no changes and full + documentation. The docstrings on the functions match the documentation if you lookup + the 3.7 API documentation. Users must manually manage the GIL when using this API + layer. + +2. An API implementation layer which knows about things like interpreters and type + symbol tables. Users must know how to manipulate the GIL and how the two garbage + collectors work with each other to add to this layer. + +3. A public API layer. Details like managing the GIL or messing with garbage collection + in general not leak out to this layer. + + + +### Interpreters + +Interpreters are global objects. After initialize!, there is a main interpreter which +is used automatically from any thread. Access to the interpreters relies on thread +local variables and interpreter locking so you can do things like: + +```clojure +(with-interpreter interpreter + some code) +``` + +This type of thing grabs the GIL if it hasn't already been claimed by the current thread +and off you go. When the code is finished, it saves the interpreter thread state back +into a global atom and thus releases the GIL. Interpreters have both shared and +per-interpreter state named `:shared-state` and `:interpreter-state` respectively. +Shared state would be the global type symbol table. Interpreter state contains things +like a map of objects to their per-interpreter python bridging class. + + +### Garbage Collection + +The system uses the tech.resource library to attach a GC hook to appropriate java object +that releases the associated python object if the java object goes out of scope. +Bridges use a similar technique to unregister the bridge on destruction of their python +counterpar. There should be no need for manual addref/release calls in any user code. + + +### Copying Vs. Mirroring + + +Objects either in python or in java may be either copied or mirrored into the other +ecosystem. Mirroring allows sharing complex and potentially changing datastructures +while copying allows a cleaner partitioning of concerns and frees both garbage +collection systems to act more independently. Numeric buffers that have a direct +representation as a C-ptr (the datatype native-buffer type) have a zero-copy pathway +via numpy. + + + ## Resources * [libpython C api](https://docs.python.org/3.7/c-api/index.html#c-api-index) * [spacy-cpp](https://github.com/d99kris/spacy-cpp) * [base classes for python protocols](https://docs.python.org/3.7/library/collections.abc.html#collections-abstract-base-classes) -## +## ## Usage ```console -sudo apt install libpython3.7-dev +sudo apt install libpython3.6-dev ``` ```clojure diff --git a/java/libpython_clj/jna/JVMBridge.java b/java/libpython_clj/jna/JVMBridge.java index 30e16b5..5906057 100644 --- a/java/libpython_clj/jna/JVMBridge.java +++ b/java/libpython_clj/jna/JVMBridge.java @@ -4,11 +4,14 @@ import com.sun.jna.*; -public interface JVMBridge +public interface JVMBridge extends AutoCloseable { public Pointer getAttr(String name); public void setAttr(String name, Pointer val); public String[] dir(); public Object interpreter(); public Object wrappedObject(); + // Called from python when the python mirror is deleted. + // This had better not throw anything. + public default void close() {} } diff --git a/project.clj b/project.clj index ab906bf..d7438bf 100644 --- a/project.clj +++ b/project.clj @@ -5,7 +5,6 @@ :url "https://www.eclipse.org/legal/epl-2.0/"} :dependencies [[org.clojure/clojure "1.10.1-beta2"] [techascent/tech.datatype "4.0-alpha36"] - [camel-snake-kebab "0.4.0"] - [potemkin "0.4.4"]] + [camel-snake-kebab "0.4.0"]] :repl-options {:init-ns clj-libpython.core} :java-source-paths ["java"])