Skip to content

Commit

Permalink
Edited without gettting latest.
Browse files Browse the repository at this point in the history
  • Loading branch information
cnuernber committed May 25, 2019
1 parent 4993940 commit a88697c
Show file tree
Hide file tree
Showing 3 changed files with 67 additions and 5 deletions.
64 changes: 62 additions & 2 deletions README.md
Expand Up @@ -10,18 +10,78 @@ advantages over a [10,000 line C binding layer](https://github.com/ninia/jep/tre
...if it works.


## Key Design Points


### Code Organization


There are 3 rough sections of code:
1. A JNA layer which is a 1-1 mapping most of the C API with no changes and full
documentation. The docstrings on the functions match the documentation if you lookup
the 3.7 API documentation. Users must manually manage the GIL when using this API
layer.

2. An API implementation layer which knows about things like interpreters and type
symbol tables. Users must know how to manipulate the GIL and how the two garbage
collectors work with each other to add to this layer.

3. A public API layer. Details like managing the GIL or messing with garbage collection
in general not leak out to this layer.



### Interpreters

Interpreters are global objects. After initialize!, there is a main interpreter which
is used automatically from any thread. Access to the interpreters relies on thread
local variables and interpreter locking so you can do things like:

```clojure
(with-interpreter interpreter
some code)
```

This type of thing grabs the GIL if it hasn't already been claimed by the current thread
and off you go. When the code is finished, it saves the interpreter thread state back
into a global atom and thus releases the GIL. Interpreters have both shared and
per-interpreter state named `:shared-state` and `:interpreter-state` respectively.
Shared state would be the global type symbol table. Interpreter state contains things
like a map of objects to their per-interpreter python bridging class.


### Garbage Collection

The system uses the tech.resource library to attach a GC hook to appropriate java object
that releases the associated python object if the java object goes out of scope.
Bridges use a similar technique to unregister the bridge on destruction of their python
counterpar. There should be no need for manual addref/release calls in any user code.


### Copying Vs. Mirroring


Objects either in python or in java may be either copied or mirrored into the other
ecosystem. Mirroring allows sharing complex and potentially changing datastructures
while copying allows a cleaner partitioning of concerns and frees both garbage
collection systems to act more independently. Numeric buffers that have a direct
representation as a C-ptr (the datatype native-buffer type) have a zero-copy pathway
via numpy.



## Resources

* [libpython C api](https://docs.python.org/3.7/c-api/index.html#c-api-index)
* [spacy-cpp](https://github.com/d99kris/spacy-cpp)
* [base classes for python protocols](https://docs.python.org/3.7/library/collections.abc.html#collections-abstract-base-classes)

##
##

## Usage

```console
sudo apt install libpython3.7-dev
sudo apt install libpython3.6-dev
```

```clojure
Expand Down
5 changes: 4 additions & 1 deletion java/libpython_clj/jna/JVMBridge.java
Expand Up @@ -4,11 +4,14 @@
import com.sun.jna.*;


public interface JVMBridge
public interface JVMBridge extends AutoCloseable
{
public Pointer getAttr(String name);
public void setAttr(String name, Pointer val);
public String[] dir();
public Object interpreter();
public Object wrappedObject();
// Called from python when the python mirror is deleted.
// This had better not throw anything.
public default void close() {}
}
3 changes: 1 addition & 2 deletions project.clj
Expand Up @@ -5,7 +5,6 @@
:url "https://www.eclipse.org/legal/epl-2.0/"}
:dependencies [[org.clojure/clojure "1.10.1-beta2"]
[techascent/tech.datatype "4.0-alpha36"]
[camel-snake-kebab "0.4.0"]
[potemkin "0.4.4"]]
[camel-snake-kebab "0.4.0"]]
:repl-options {:init-ns clj-libpython.core}
:java-source-paths ["java"])

0 comments on commit a88697c

Please sign in to comment.