-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Seeking feedback on new API for adaptive acquisition #654
Comments
example: from pycromanager import Acquisition, multi_d_acquisition_events, AcqNotification
import time
import numpy as np
events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)
print_notification_fn = lambda x: print(x.to_json())
start = time.time()
with Acquisition(directory='C:\\Users\\henry\\Desktop\\data', name='acq', notification_callback_fn=print_notification_fn,
show_display=False) as acq:
start = time.time()
future = acq.acquire(events)
future.await_execution({'time': 5}, AcqNotification.Hardware.POST_HARDWARE)
print('time point 5 post hardware')
image = future.await_image_saved({'time': 5}, return_image=True)
print('time point 5 image saved ', time.time() - start, '\t\tmean image value: ', np.mean(image))
images = future.await_image_saved([{'time': 7}, {'time': 8}, {'time': 9}], return_image=True)
assert (len(images) == 3)
for image in images:
print('mean of image in stack: ', np.mean(image))
# Make sure the returned images were the correct ones
on_disk = [acq.get_dataset().read_image(time=t) for t in [7, 8, 9]]
assert all([np.all(on_disk[i] == images[i]) for i in range(3)]) @ieivanov Interested in trying this out and seeing what you think? I think that has the potential to greatly simplify the way you currently do this in #567. |
@wl-stepp maybe this is of interest to you as well? |
Sounds interesting, thanks. Will have a closer look and think about it. |
Hey @henrypinkard, thanks for your work on this. I'm a bit tied up with work on a manuscript for the next two months. I expect I won't be doing much software development until then, but I'll be eager to try it after! |
@ieivanov No worries, good luck with the paper! |
Feels natural to explicitly get the
Milestone sounds intuitive. Acquisition milestone maybe if it's needed to be more explicit? |
Thanks, this is super helpful!
Right now its set up so that the So it might be simplest to say that if you're adding events from a hook, you cant get access to these objects. Do you have a specific use case you're thinking of here?
The acquisition won't end until you exit the
Alternatively, you can forgo the acq = Acquisition(...
# stuff
acq.mark_finished()
The other idea I had was "phase". What do you think of |
@henrypinkard just chiming in since this is crossing paths with my work on this PR. The notification system would be ideal for me to monitor the acquisition progression and show it on the UI interactively. I tried testing the snippet above with from pycromanager import Acquisition, multi_d_acquisition_events, AcqNotification, start_headless
from pymmcore_plus import find_micromanager
import time, os
import numpy as np
events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)
mm_path = find_micromanager()
config_file = os.path.join(mm_path, 'MMConfig_demo.cfg')
print(config_file)
start_headless(mm_app_path=mm_path, config_file=config_file)
start = time.time()
with Acquisition(directory='./test_rec', name='acq_notification', show_display=False) as acq:
start = time.time()
future = acq.acquire(events)
future.await_execution({'time': 5}, AcqNotification.Acquisition.ACQ_EVENTS_FINISHED)
print('Acquisition finished') But I got the following traceback
I imagine this is due to the fact that the implementation is still missing. One thing I would like to have though - and this probably extends to all hook functions - would be the possibility to have a list of callables rather than a single one; in my case, it's to make sure that the basic acquisition widget functionalities can be expanded in case a widget that uses PycroManager can implement another hook function of the same type. |
Well, our event-driven acquisitions would add events and ideally we don't have to specify how many beforehand. Then if something else is done with the futures, say every 20 frames, it might not be possible to await it if the event was not there in the beginning. Do I get that correctly?
|
@jacopoabramo no problem to add a list of callables as an option as well. And yes there are definitely still some bugs to work out here...This isn't fully tested or finalized yet |
@wl-stepp This is great feedback! I think I will change it to milestone
Maybe the solution here is to remove the version of the hook function where you get access to the event queue, and instead you'd just accomplish the same thing by calling A big motivation for this API is to make things like EDA easier, such that you can write a single script specifying what you're trying to do that makes the feedback loops more clear, while still getting performance benefits of parallelization, simple scripting, etc. For example: with Acquisition...as acq:
while True:
af = acq.acquire(multi_d_acquisition_events(num_time_points=1))
image = af.await_image_saved(return_image=True, time=0)
# analyze the image
if interesting_phenotype:
# run a fast timelapse
events = multi_d_acquisition_events(num_time_points=10, time_interval_s=0)
af = acq.acquire(events)
else:
time.sleep(10) # slow timelapse So the hope would be that you can accomplish everything you need to without having to add events in the hook. What do you think? |
I've been playing around a bit using the (I understand currently unfinished) notification system. I just have some pro-forma notes/questions, mostly related to what I'm currently implementing (a.k.a. pycromanager as backend for ImSwitch). Pretty sure this is all stuff that's already planned but it would be hugely of help to have them soon. Time points notifications using
|
Thanks for the feedback on notifications @jacopoabramo !
Unfortunately this is not possible with the current APIs. In this case of a timelapse with no delay in between, the acquisition engine runs a sequence acquisition, which allows the camera to capture frames faster. However, there is simply a start sequence API call and a stop sequence one, without any intermediate calls or updates from the hardware coming from individual frames having the exposure finish. So, confusingly, the notifications are currently different depending on whether the acquisition engine is executing a series of snaps or a sequence. The new camera API (micro-manager/mmCoreAndDevices#243) can address this and provide these notifications in all cases, but there will still likely be many cameras operating on the old API for some time. I think I'm going to change the notification types to
I'm confused what you mean for this one. Could you not just accomplish this by not passing labels for the positions?
That's because the notifications are printing the That XYZ positions feature is designed to support the case where you move to different positions in 3D and then run z-stacks. Here, there are no Z-stacks, so every position has a {'axes': {'time': 2, 'position': 'P1', 'z': 0}, 'x': 100, 'y': 0, 'z': 50},
{'axes': {'time': 2, 'position': 'P2', 'z': 0}, 'x': 0, 'y': 100, 'z': 50},
{'axes': {'time': 2, 'position': 'P3', 'z': 0}, 'x': 100, 'y': 100, 'z': 100}] |
Got it; I think I managed somehow to solve this issue in my implementation but I honestly can't recall how I dealt with it.
The reason why I asked this was because in my ImSwitch PR I wanted to monitor an acquisition over XYZ using indexes that could be used by a Qt progress bar; in the end I opted for generating the indexes directly into ImSwitch and parse the incoming position label to retrieve the correct index (the assumption being that pycro-manager would perform each step sequentially), so it's not important anymore.
Can't promise I'll have the time to prepare a PR but I'll have a look at the code for sure |
All sounds good. No pressure on making a PR -- only if you want to |
Following up on #567
Making an asynchronous equivalent of what hooks, image processors, and image saved callbacks do right now would make it much easier to express complex experiments in a readable way in pycromanager. It would provide the benefits of the parallelized backend (i.e. hardware and images handled on separate threads), but would allow user code to be readable, sequential commands.
I think I know approximately how to do this on the backend, but it would be great to get some feedback on what the API should look like. This is powered by adding to the acquisition engine a new feature I'm calling "notifications": asynchronous updates about what is going on on its different threads (e.g. finished the
pre_hardware_hook
for{'time': 3, 'z': 4}
, acquired image'{'time': 5, 'z': 6}'
, etc.).One way of doing this from the user perspective, when you call
acq.acquire(events)
, it would now return anAcquisitionFuture
object that allows you to wait on these various things happening. Example:Could also have this for images (and this might even be more useful than hooks for data-adaptive acquisitions):
I think it probably would not make sense to have a
await_image_acquired(return_image=False)
, because this would mean all images would need to be hung onto in RAM in case you called this function later.Behind the scenes, when you call
acq.acquire
, theAcquisitionFuture
is dynamically generated with knowledge of all possible "milestones" it could be awaiting (i.e. all points in the acquisition cycle for allaxes
combos present in the events). As long as theAcquisitionFuture
is not garbage collected, it is monitoring the notifications coming from the acquisition engine, so that it can knows that a notification has been received even if you ask it toawait_execution
after the notification already happened.AcquisitionFuture
s going out of scope and getting garbage collected is how you keep notifications from creating too many objects and clogging up memoryHow should this API look?
AcquisitionFuture
object? One advantage I see of the latter would be that it might be able to be dynamically generated in some smart way to have attributes corresponding to all the things it can wait on, likeaf.time
,af.z
etc. Is this useful or are there better things that could be done?milestone
to express "point in the hardware control and image acquisition cycle"?The text was updated successfully, but these errors were encountered: