Seeking feedback on new API for adaptive acquisition

[henrypinkard]

Following up on #567

Making an asynchronous equivalent of what hooks, image processors, and image saved callbacks do right now would make it much easier to express complex experiments in a readable way in pycromanager. It would provide the benefits of the parallelized backend (i.e. hardware and images handled on separate threads), but would allow user code to be readable, sequential commands.

I think I know approximately how to do this on the backend, but it would be great to get some feedback on what the API should look like. This is powered by adding to the acquisition engine a new feature I'm calling "notifications": asynchronous updates about what is going on on its different threads (e.g. finished the pre_hardware_hook for {'time': 3, 'z': 4}, acquired image '{'time': 5, 'z': 6}', etc.).

One way of doing this from the user perspective, when you call acq.acquire(events), it would now return an AcquisitionFuture object that allows you to wait on these various things happening. Example:

with Acquisition(...) as acq:
   events = multi_d_acquisition_events(num_time_points=5)
   af = acq.acquire(events)
   # now pause until the 3rd time point is about  to be acquired
   # unlike a hook, this call will not block the acq engine from proceeding after it reaches this point
   # it just blocks the main thread from proceeding until this has happened
   af.await_execution(milestone='pre_hardware', time=3) 

  # or a different way of doing this would be to only keep the AcquisitionFuture internally
  # and just call 
  acq.await_execution(milestone='pre_hardware', time=3) 

Could also have this for images (and this might even be more useful than hooks for data-adaptive acquisitions):

with Acquisition(...) as acq:
   events = multi_d_acquisition_events(num_time_points=5)
   af = acq.acquire(events)
   image = af.await_image_saved(return_image=True, time=3)
   # process the image here and decide what events to send next

I think it probably would not make sense to have a await_image_acquired(return_image=False), because this would mean all images would need to be hung onto in RAM in case you called this function later.

Behind the scenes, when you call acq.acquire, the AcquisitionFuture is dynamically generated with knowledge of all possible "milestones" it could be awaiting (i.e. all points in the acquisition cycle for all axes combos present in the events). As long as the AcquisitionFuture is not garbage collected, it is monitoring the notifications coming from the acquisition engine, so that it can knows that a notification has been received even if you ask it to await_execution after the notification already happened. AcquisitionFutures going out of scope and getting garbage collected is how you keep notifications from creating too many objects and clogging up memory

How should this API look?

• acq.awaitExecution() or explicitly get an AcquisitionFuture object? One advantage I see of the latter would be that it might be able to be dynamically generated in some smart way to have attributes corresponding to all the things it can wait on, like af.time, af.z etc. Is this useful or are there better things that could be done?
• Is there a better keyword than milestone to express "point in the hardware control and image acquisition cycle"?
• Other ideas or improvements?

[henrypinkard]

example:

from pycromanager import Acquisition, multi_d_acquisition_events, AcqNotification
import time
import numpy as np

events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)

print_notification_fn = lambda x: print(x.to_json())

start = time.time()
with Acquisition(directory='C:\\Users\\henry\\Desktop\\data', name='acq', notification_callback_fn=print_notification_fn,
                 show_display=False) as acq:
    start = time.time()
    future = acq.acquire(events)

    future.await_execution({'time': 5}, AcqNotification.Hardware.POST_HARDWARE)
    print('time point 5 post hardware')

    image = future.await_image_saved({'time': 5}, return_image=True)
    print('time point 5 image saved ', time.time() - start, '\t\tmean image value: ', np.mean(image))

    images = future.await_image_saved([{'time': 7}, {'time': 8}, {'time': 9}], return_image=True)
    assert (len(images) == 3)
    for image in images:
        print('mean of image in stack: ', np.mean(image))

# Make sure the returned images were the correct ones
on_disk = [acq.get_dataset().read_image(time=t) for t in [7, 8, 9]]
assert all([np.all(on_disk[i] == images[i]) for i in range(3)])

@ieivanov Interested in trying this out and seeing what you think? I think that has the potential to greatly simplify the way you currently do this in #567.

[henrypinkard]

@wl-stepp maybe this is of interest to you as well?

[wl-stepp]

Sounds interesting, thanks. Will have a closer look and think about it.

[ieivanov]

Hey @henrypinkard, thanks for your work on this. I'm a bit tied up with work on a manuscript for the next two months. I expect I won't be doing much software development until then, but I'll be eager to try it after!

[henrypinkard]

@ieivanov No worries, good luck with the paper!

[wl-stepp]

* acq.awaitExecution() or explicitly get an `AcquisitionFuture` object? One advantage I see of the latter would be that it might be able to be dynamically generated in some smart way to have attributes corresponding to all the things it can wait on, like `af.time`, `af.z` etc. Is this useful or are there better things that could be done?

Feels natural to explicitly get the AcquisitionFuture. How does it work if acquisition events are added later during a hook, or after waiting for a milestone? Can I get a new updated future that allows me to wait for the newly added event?
I like that it doesn't block, but what happens if the acquisition ends while I'm doing things on something I waited for? Is there a way of keeping the acquisition active even if there are no more events?

* Is there a better keyword than `milestone` to express "point in the hardware control and image acquisition cycle"?

Milestone sounds intuitive. Acquisition milestone maybe if it's needed to be more explicit?

[henrypinkard]

Thanks, this is super helpful!

How does it work if acquisition events are added later during a hook, or after waiting for a milestone? Can I get a new updated future that allows me to wait for the newly added event?

Right now its set up so that the AcquisitionFuture can only await milestones associated with the events that were passed to acq.acquire, the reason being that if every AcquisitionFuture had to have the ability to await every notification from the acquisition engine, the acquisition would have to keep track of every notification ever received and I could so this causing problems on very long acquisitions.

So it might be simplest to say that if you're adding events from a hook, you cant get access to these objects. Do you have a specific use case you're thinking of here?

I like that it doesn't block, but what happens if the acquisition ends while I'm doing things on something I waited for?

The acquisition won't end until you exit the with Acquisition... block, so as long as you do processing there it shouldn't cause issues. The other case is if the acquisition gets aborted. I think it might make sense to then having these awaiting calls throw an exception if the thing being awaited never arrives

Is there a way of keeping the acquisition active even if there are no more events?

Alternatively, you can forgo the with... syntax and do :

acq = Acquisition(...

# stuff

acq.mark_finished()

Milestone sounds intuitive. Acquisition milestone maybe if it's needed to be more explicit?

The other idea I had was "phase". What do you think of phase or acquisition_phase?

[jacopoabramo]

@henrypinkard just chiming in since this is crossing paths with my work on this PR. The notification system would be ideal for me to monitor the acquisition progression and show it on the UI interactively. I tried testing the snippet above with Acquisition.ACQ_EVENTS_FINISHED with the following snippet:

from pycromanager import Acquisition, multi_d_acquisition_events, AcqNotification, start_headless
from pymmcore_plus import find_micromanager
import time, os
import numpy as np

events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)

mm_path = find_micromanager()
config_file = os.path.join(mm_path, 'MMConfig_demo.cfg')

print(config_file)

start_headless(mm_app_path=mm_path, config_file=config_file)

start = time.time()
with Acquisition(directory='./test_rec', name='acq_notification', show_display=False) as acq:
    start = time.time()
    future = acq.acquire(events)

    future.await_execution({'time': 5}, AcqNotification.Acquisition.ACQ_EVENTS_FINISHED)
    print('Acquisition finished')

But I got the following traceback

Traceback (most recent call last):
  File "C:\git\sandbox\pycromanager_notification_test.py", line 23, in <module>
    future.await_execution({'time': 5}, AcqNotification.Acquisition.ACQ_EVENTS_FINISHED)
  File "C:\Users\jacop\AppData\Local\Programs\Python\Python310\lib\site-packages\pycromanager\acq_future.py", line 58, in await_execution
    notification = AcqNotification(None, axes, phase)
  File "C:\Users\jacop\AppData\Local\Programs\Python\Python310\lib\site-packages\pycromanager\acquisition\acq_eng_py\main\acq_notification.py", line 58, in __init__ 
    raise ValueError("Unknown phase")
ValueError: Unknown phase

I imagine this is due to the fact that the implementation is still missing. One thing I would like to have though - and this probably extends to all hook functions - would be the possibility to have a list of callables rather than a single one; in my case, it's to make sure that the basic acquisition widget functionalities can be expanded in case a widget that uses PycroManager can implement another hook function of the same type.

[wl-stepp]

So it might be simplest to say that if you're adding events from a hook, you cant get access to these objects. Do you have a specific use case you're thinking of here?

Well, our event-driven acquisitions would add events and ideally we don't have to specify how many beforehand. Then if something else is done with the futures, say every 20 frames, it might not be possible to await it if the event was not there in the beginning. Do I get that correctly?

phase sounds to me more like an extended thing, not a point in the acquisition. But not a native speaker myself...

[henrypinkard]

One thing I would like to have though - and this probably extends to all hook functions - would be the possibility to have a list of callables rather than a single one; in my case, it's to make sure that the basic acquisition widget functionalities can be expanded in case a widget that uses PycroManager can implement another hook function of the same type.

@jacopoabramo no problem to add a list of callables as an option as well.

And yes there are definitely still some bugs to work out here...This isn't fully tested or finalized yet

[henrypinkard]

phase sounds to me more like an extended thing, not a point in the acquisition. But not a native speaker myself...

@wl-stepp This is great feedback! I think I will change it to milestone

Well, our event-driven acquisitions would add events and ideally we don't have to specify how many beforehand. Then if something else is done with the futures, say every 20 frames, it might not be possible to await it if the event was not there in the beginning. Do I get that correctly?

Maybe the solution here is to remove the version of the hook function where you get access to the event queue, and instead you'd just accomplish the same thing by calling acq.acquire within the hook.

A big motivation for this API is to make things like EDA easier, such that you can write a single script specifying what you're trying to do that makes the feedback loops more clear, while still getting performance benefits of parallelization, simple scripting, etc. For example:

with Acquisition...as acq:
   while True:
     af = acq.acquire(multi_d_acquisition_events(num_time_points=1))
     image = af.await_image_saved(return_image=True, time=0)

     # analyze the image
     if interesting_phenotype:
       # run a fast timelapse
        events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)
        af = acq.acquire(events)
     else:
        time.sleep(10) # slow timelapse

So the hope would be that you can accomplish everything you need to without having to add events in the hook. What do you think?

[jacopoabramo]

I've been playing around a bit using the (I understand currently unfinished) notification system. I just have some pro-forma notes/questions, mostly related to what I'm currently implementing (a.k.a. pycromanager as backend for ImSwitch). Pretty sure this is all stuff that's already planned but it would be hugely of help to have them soon.

---

Time points notifications using POST_EXPOSURE phase

Reference script:

from pycromanager import start_headless, Acquisition, multi_d_acquisition_events, AcqNotification
from pymmcore_plus import find_micromanager 
import numpy as np

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

mm_app_path = find_micromanager()
logger.info("Found Micro-Manager at: " + mm_app_path)
config_file = mm_app_path + "/MMConfig_demo.cfg"

save_dir = "./data"

# Start the Java process
start_headless(mm_app_path, config_file)

def notification_fn(msg: AcqNotification):
    print(msg.to_json())

def process(image, metadata):
    pass

with Acquisition(notification_callback_fn=notification_fn, image_process_fn=process, show_display=False, debug=False) as acq:
    events = multi_d_acquisition_events(
        num_time_points=10,
    )
    acq.acquire(events)

In this case, the notification_fn hook produces the following:

{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_started'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': [{'time': 0}, {'time': 1}, {'time': 2}, {'time': 3}, {'time': 4}, {'time': 5}, {'time': 6}, {'time': 7}, {'time': 8}, {'time': 9}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': [{'time': 0}, {'time': 1}, {'time': 2}, {'time': 
3}, {'time': 4}, {'time': 5}, {'time': 6}, {'time': 7}, {'time': 8}, {'time': 9}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_sequence_started', 'id': [{'time': 0}, {'time': 1}, {'time': 2}, {'time': 3}, {'time': 4}, {'time': 5}, {'time': 6}, {'time': 7}, {'time': 8}, {'time': 9}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_events_finished'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Image'>, 'phase': 'data_sink_finished'}

I'm not sure if this is intentional but, if possible, I would like to have a POST_EXPOSURE notification with it's relative integer ID, for each captured image.

---

Adding integer ID for labeled positions

Reference script is similar, only change is in the generated events:

xy_positions = np.array([[0, 0], [100, 0], [0, 100], [100, 100]])
labels = ["P0", "P1", "P2", "P3"]

with Acquisition(notification_callback_fn=notification_fn, image_process_fn=process, show_display=False, debug=False) as acq:
    events = multi_d_acquisition_events(
        num_time_points=3,
        xy_positions=xy_positions,
        position_labels=labels,
    )
    acq.acquire(events)

Notification output is:

{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_started'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 0, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 0, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 0, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 0, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 0, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 0, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 0, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 0, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 0, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 0, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 0, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 0, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 0, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 0, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 0, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 0, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 1, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 1, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 1, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 1, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 1, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 1, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 1, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 1, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 1, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 1, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 1, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 1, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 1, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 1, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 1, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 1, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 2, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 2, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 2, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 2, 'position': 'P0'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 2, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 2, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 2, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 2, 'position': 'P1'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 2, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 2, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 2, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 2, 'position': 'P2'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': {'time': 2, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': {'time': 2, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_snap', 'id': {'time': 2, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'post_exposure', 'id': {'time': 2, 'position': 'P3'}}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_events_finished'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Image'>, 'phase': 'data_sink_finished'}

This is more coherent with what I expect from the previous use-case; what I would like to have is an integer ID of the current captured position (i.e. position_id), in order to better monitor the execution of the acquisition.

---

I also noticed that when using xyz_positions like this:

xyz_positions = np.array([[0, 0, 50], [100, 0, 50], [0, 100, 50], [100, 100, 100]])
labels = ["P0", "P1", "P2", "P3"]

with Acquisition(notification_callback_fn=notification_fn, image_process_fn=process, show_display=False, debug=False) as acq:
    events = multi_d_acquisition_events(
        num_time_points=3,
        xyz_positions=xyz_positions,
        position_labels=labels,
    )
    acq.acquire(events)

The notification output looks like this:

{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_started'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': [{'z': 0, 'position': 'P0', 'time': 0}, {'z': 0, 'position': 'P0', 'time': 1}, {'z': 0, 'position': 'P0', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': [{'z': 0, 'position': 'P0', 'time': 0}, {'z': 0, 'position': 'P0', 
'time': 1}, {'z': 0, 'position': 'P0', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_sequence_started', 'id': [{'z': 0, 'position': 'P0', 'time': 0}, {'z': 0, 'position': 'P0', 'time': 1}, {'z': 0, 'position': 'P0', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': [{'z': 0, 'position': 'P1', 'time': 0}, {'z': 0, 'position': 'P1', 'time': 1}, {'z': 0, 'position': 'P1', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': [{'z': 0, 'position': 'P1', 'time': 0}, {'z': 0, 'position': 'P1', 
'time': 1}, {'z': 0, 'position': 'P1', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_sequence_started', 'id': [{'z': 0, 'position': 'P1', 'time': 0}, {'z': 0, 'position': 'P1', 'time': 1}, {'z': 0, 'position': 'P1', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': [{'z': 0, 'position': 'P2', 'time': 0}, {'z': 0, 'position': 'P2', 'time': 1}, {'z': 0, 'position': 'P2', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': [{'z': 0, 'position': 'P2', 'time': 0}, {'z': 0, 'position': 'P2', 
'time': 1}, {'z': 0, 'position': 'P2', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_sequence_started', 'id': [{'z': 0, 'position': 'P2', 'time': 0}, {'z': 0, 'position': 'P2', 'time': 1}, {'z': 0, 'position': 'P2', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'pre_hardware', 'id': [{'z': 0, 'position': 'P3', 'time': 0}, {'z': 0, 'position': 'P3', 'time': 1}, {'z': 0, 'position': 'P3', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Hardware'>, 'phase': 'post_hardware', 'id': [{'z': 0, 'position': 'P3', 'time': 0}, {'z': 0, 'position': 'P3', 
'time': 1}, {'z': 0, 'position': 'P3', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Camera'>, 'phase': 'pre_sequence_started', 'id': [{'z': 0, 'position': 'P3', 'time': 0}, {'z': 0, 'position': 'P3', 'time': 1}, {'z': 0, 'position': 'P3', 'time': 2}]}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Acquisition'>, 'phase': 'acq_events_finished'}
{'type': <class 'pycromanager.acquisition.acq_eng_py.main.acq_notification.AcqNotification.Image'>, 'phase': 'data_sink_finished'}

Not sure if it is intentional or a bug, but apparently the z ID never changes.

---

EDIT: I would contribute myself to these changes which I believe should be relatively small but in all honesty I wouldn't know where to start and some pointers are appreciated.

[henrypinkard]

Thanks for the feedback on notifications @jacopoabramo !

I'm not sure if this is intentional but, if possible, I would like to have a POST_EXPOSURE notification with it's relative integer ID, for each captured image.

Unfortunately this is not possible with the current APIs. In this case of a timelapse with no delay in between, the acquisition engine runs a sequence acquisition, which allows the camera to capture frames faster. However, there is simply a start sequence API call and a stop sequence one, without any intermediate calls or updates from the hardware coming from individual frames having the exposure finish.

So, confusingly, the notifications are currently different depending on whether the acquisition engine is executing a series of snaps or a sequence. The new camera API (micro-manager/mmCoreAndDevices#243) can address this and provide these notifications in all cases, but there will still likely be many cameras operating on the old API for some time. I think I'm going to change the notification types to pre_snap \ post_snap and pre_sequence \ post_sequence to avoid this confusion

This is more coherent with what I expect from the previous use-case; what I would like to have is an integer ID of the current captured position (i.e. position_id), in order to better monitor the execution of the acquisition.

I'm confused what you mean for this one. Could you not just accomplish this by not passing labels for the positions?

Not sure if it is intentional or a bug, but apparently the z ID never changes.

That's because the notifications are printing the axes rather than the full event, since sending the full event in the notification might come with a performance penalty, and presumably the user already has the event anyway.

That XYZ positions feature is designed to support the case where you move to different positions in 3D and then run z-stacks. Here, there are no Z-stacks, so every position has a z index of 0. I think it would be reasonable to change the multi_d_acq_events function to not include z in the axes in the case where z-stacks are not in use if you have any interest in PRing that

 {'axes': {'time': 2, 'position': 'P1', 'z': 0}, 'x': 100, 'y': 0, 'z': 50},
 {'axes': {'time': 2, 'position': 'P2', 'z': 0}, 'x': 0, 'y': 100, 'z': 50},
 {'axes': {'time': 2, 'position': 'P3', 'z': 0}, 'x': 100, 'y': 100, 'z': 100}]

[jacopoabramo]

@henrypinkard

Unfortunately this is not possible with the current APIs.

Got it; I think I managed somehow to solve this issue in my implementation but I honestly can't recall how I dealt with it.

I'm confused what you mean for this one. Could you not just accomplish this by not passing labels for the positions?

The reason why I asked this was because in my ImSwitch PR I wanted to monitor an acquisition over XYZ using indexes that could be used by a Qt progress bar; in the end I opted for generating the indexes directly into ImSwitch and parse the incoming position label to retrieve the correct index (the assumption being that pycro-manager would perform each step sequentially), so it's not important anymore.

I think it would be reasonable to change the multi_d_acq_events function to not include z in the axes in the case where z-stacks are not in use if you have any interest in PRing that

Can't promise I'll have the time to prepare a PR but I'll have a look at the code for sure

[henrypinkard]

All sounds good. No pressure on making a PR -- only if you want to