ZMQServer to micromanager

[henrypinkard]

This version automatically gets all non internal Micromanager classes and as well as plugins (at least from netbeans, haven't tried from built version which might be a good check). I also added a generic mechanism for accessing a plugin by its classpath. Still needs a mechanism for enabling from tools, maybe you can do this more easily than I (line 898 of MMStudio).

You'll need to pull new version of Pygellan. Take a look at this (incomplete) example:

[nicost]

I just pushed code that starts (and stops) the zmq server as I think it should. However, the Python side of things does not work for me. I have pygellan 0.1.6 installed (which is what I get from pip install pygellan). I figured that I need to delete the old compiled version of the Magellan plugin, since it starts the ZMQ server by itself (and most likely holds on to the port).

When I do:

from pygellan.acquire import MagellanBridge
bridge = MagellanBridge();

I see the following on the Java side:
org.json.JSONException: JSONObject["classpath"] not found. in Thread[ZMQ Server master,6,main]
[ ] at org.json.JSONObject.get(JSONObject.java:285)
[ ] at org.json.JSONObject.getString(JSONObject.java:411)
[ ] at org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:86)
[ ] at org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:56)
[ ] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ ] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ ] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ ] at java.lang.Thread.run(Thread.java:748)

Ideas?

[henrypinkard]

You probably need to delete the existing Magellan jar, because unless you rebuilt it it is still running one too.

Also you should pull the latest pygellan source code (I didn't want to push a new version to PyPi yet so as not to disrupt existing users). You can either remove the pip installed version and add the source code to your path, or do a pip install -e . when in the top level folder of pygellan to make the local code called by python

[henrypinkard]

Also I changed the name of MagellanBridge to PygellanBridge, so that'll be one way to verify its working

[henrypinkard]

Forgot to add link to the new example above. Its here: https://github.com/henrypinkard/Pygellan/blob/master/examples/micro-manager_api.py

[nicost]

pip install -e

results in:
ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /Users/nico/git/Pygellan/pygellan

Basically I have no idea how to get this new pygellan code to work in Pycharm.

[bryantChhun]

did you forget the "." ? you'll want

pip install -e .

[henrypinkard]

On second thought I'm not sure you actually need to do that. If you set up your pycharm project with the root folder of the pygellan git repo as the project folder it should just work I think

[nicost]

After lots of environment mangling (have to learn more about Pycharm;) managed to get it to work. Sweet! Managed to do:

from pygellan.acquire import PygellanBridge
bridge = PygellanBridge()
mm = bridge.get_studio()
mmc = bridge.get_core()
mmc.set_exposure("Camera", 100)
mm.live().snap(True)

and all worked (more or less). Questions/issues I encountered:

• mmc.set_exposure(100) did not work. Is there no overloading in Python?
• Is there a switch yet to turn on/off name wrangling? Could that best be as a parameter to PygellanBridge?
• Doing repeated mm.live().snap(True), it turned out that each call has an exception, but those are only shown after a while. I guess that the buffering happens in my environment (the buffering is kind of annoying)? I could not find the source of the exception in thge Java environment. Python reports:

<pygellan.acquire.JavaObjectShadow object at 0x113fac610>]
Exception ignored in: <function JavaObjectShadow.__del__ at 0x113ee7cb0>
Traceback (most recent call last):
  File "/Users/nico/git/Pygellan/pygellan/acquire.py", line 155, in __del__
    if reply_json['type'] == 'exception':
KeyError: ('type',)

• How does one get objects for other classes? is that the "construct_java_object_from_classpath" function? OK to rename to "get_java_object"?

[henrypinkard]

Nice!

• There's no direct analog of overloading based on static types in Java, but there are ways around this. You basically just have to inspect the argument runtime types and dispatch to different functions. Not impossible but will take a little thought. I made an issue: Operator overloading based on argument types henrypinkard/Pygellan#26
• Yes, bridge = PygellanBridge(convert_camel_case=False) should already work in the version you have
• This was a bug in pygellan not java. I fixed it, but now another one cropped up. Still working on it
• Yes, exactly. Just renamed it. If the the object is core or studio, it grabs the existing instance through micro-manager api. If its something else (e.g. Magellan API) it instantiates a new object

sending you another PR with some small fixes now on micromanager, and pushing new stuff to pygellan as well

[henrypinkard]

Fixed the bug in the third bullet in new PR

[nicost]

Awesome! Amazing turn-around;)

• Small thing: pygellan.acquire PygellanBridge.recieve probably should be "receive"
• StrVectors do not seem to be supported, but they do come from the core. so:

mmc.getDevicePropertyNames("XY")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/Users/nico/git/Pygellan/pygellan/acquire.py", line 200, in _translate_call
    return self._deserialize_return(reply)
  File "/Users/nico/git/Pygellan/pygellan/acquire.py", line 213, in _deserialize_return
    raise Exception(json_return['value'])
Exception: Internal class accidentally exposed

Any way to deal with that elegantly?

[henrypinkard]

Good catch...I only know how to spell in binary

Another PR coming that exposes all mmcorej classes. It may be that it makes more sense to hard code in explicit conversions to python types for some of these, as I did with TaggedImage and ArrayList. For now they are ShadowClasses

Also fixed the function overloading thing (re-pull Pygellan). Turns out I implemented this a few months ago and forgot, but there was a small bug. What is not implemented though is autocomplete with all versions with different numbers of arguments. For now it just defaults to the one with the longest argument list

[nicost]

Still the same error, I think because mmcorej is now added to the apiClasses_, but not mmcorej.StrVector. Adding the line:
apiClasses_.add(StrVector.class);
solves this (but does not add the other classes in mmcorej). I guess that somehow the mmcorej package is not visible in the current classloader?

Is there a way from Python to "close" the bridge? It is easy to restart a Python console to accomplish the same thing, but it seems more elegant to close the bridge and start a new one.

[henrypinkard]

new PR for MMCoreJ problems. It wasnt loading them because they were in a jar rather than on the file system (CMMCore still worked because it was added manually as a vestige from before)

[henrypinkard]

pushed code with a destructor that shuts down all sockets and ZMQ. This will happen if the bridge gets garbage collected, or you can trigger it manuall with del bridge

henrypinkard/Pygellan@96a42fc