Localization PSF width distribution has gaps

[loekloek]

• Picasso version: 0.6.9
• Python version: Windows install/3.10.13
• Operating System: Windows

Description

After performing localization, I see strange 'holes' in my sx and sy distribution.

What I Did

I use Picasso localize to localize and fit my nd2 movie. I have a z-calibration file based on astigmatism and use the parameters I attached below. I noticed strange z distributions and investigated further. First I thought the problem might be with my calibration, however, I noticed that sx and sy distributions (the sizes of the PSF in x and y of the localizations) show that certain PSF shapes seem to not exist. You can find the histograms for the z distributions and the sx and sy distributions attached below, which I made using Picasso Filter after the localizations.

I tested it both on the Windows install and on the PyPi install and both gave the same result. Also, I see similar holes in datasets from different measurements on different days. Could there be something in the code that would 'filter' out certain PSF shapes?

Let me know if you need any extra info! Thanks for your help in advance :).

Parameters:
configtxt.txt
Sx, sy:

Z:

[rafalkowalewski1]

Hi

Could you please share with me your raw file used for the calibration? Also, did you notice the same behaviour for other datasets?

I would also appreciate if you could list the parameters you use for the calibration.

Bests,
Rafal

[loekloek]

Hi Rafal,

Thanks for the response. I notice the same behavior in other datasets, often the shape of the histogram is very similar to this one, and sometimes I get a different shape, however, always there are big parts that seem to be 'artificially' missing. I will attach the raw calibration file and the parameter list for you.

I have experimented with different calibrations and did see that sx and sy are dependent on the calibration file, which I did not expect, as I thought they were the size of the PSF of the raw localizations. Could you elaborate on what the variables mean, are the values of sx and sy that I plot already fitted on the calibration data?

Parameter list:

baseline: 102.7
sensitivity: 1.0
quantum_efficiency: 0.09
em_gain: 1
pixelsize: 160
box_size: 25
min_net_gradient: 7500
fit_method: lq-3d
ROI: null
stepize_z: 25

Raw calibration file:
231124 - z-stack2um_100nm_561nm_5_corrco_170um001-1.zip

Let me know if there is anything else you need!

Kind regards,
Loek

[rafalkowalewski1]

The columns in the .hdf5 are explained here: https://picassosr.readthedocs.io/en/latest/files.html#localization-hdf5-files

I do not know if this is specific to your user case, but in our lab we usually use the step size of 5 nm. Morevoer, the 2 um range is quite large, you probably will not to image that far away from the focal plane anyway. Box size of 25 leads to large overlaps which may impact your results. Something around 9 should work better (especially when you aim at a smaller range around z=0, for example +/ 400 nm). Adding more beads to your sample will help improve precision too.

Unfortunately, I cannot pinpoint one reason why you observe such gaps in your data. I hope that introducing such changes will help you. If not, please let me know.

Bests,
Rafal

[loekloek]

Dear Rafal,

Thanks again for the response. Our setup is unable to make smaller steps than 25 nm, unfortunately, hence the 25nm step size. I retried, on a new calibration with more beads and with the settings you recommended. This yielded the sx,sy histogram below:

You could say it looks better, however, I still see these gaps in my data. I was wondering if you have the same in your setup? Also, thanks for referring to the explanation on the sx and sy, however the description is quite brief. I understand that they represent the width and height of the PSFs of the localizations, however, I was wondering how they are processed from the point that they are detected in Picasso localize, up until I see them pop up in my plot. If I understand correctly, the sx and sy that are found in the localizations with too big of an error on the curve fit are rejected during the 3d fitting. Do you expect that that could lead to the behavior we see in this graph? Or is there another processing step that I am missing?

Let me know what you think! I would also be open to a small video call to make discussions easier if you think that could help.

Kind regards,
Loek

[loekloek]

Hi Rafal,

I was wondering if you have had the time to read the post above! Let me know if you need any additional information.

Kind regards,
Loek

[rafalkowalewski1]

Hi Loek,

Apologies for a late reply. To be honest, I do not know how to help you further, I will discuss your issue with other people in the lab and let you know once we come up with a fix.
In fact, we have encountered something similar in the past, although I am not sure how it was solved.

Regarding your latest reply, there is a filtering step, where each spot's width and height are compared to the variances in the width and height across the dataset. I am not sure, however, if that is the reason why the problem occurs. If you are interested in reading the code that does the filter, please check here:

picasso/picasso/zfit.py

Line 38 in 35c3d8e

keep_x = (locs.sx - mean_sx[locs.frame]) ** 2 < var_sx[locs.frame]

Bests,
Rafal

[loekloek]

Dear Rafal,

Thanks for the reply! I am very interested in hearing about how the similar thing you encountered was fixed.

Thanks for the code link, I checked and I think this is only responsible for creating the calibration curve, while I expect that something weird is going on in the fitting of single spots or mapping those spots onto the calibration curve. I did test this by removing this filtering step from the pipeline, but the gaps still remain.

Let me know if anything came out of your discussions!

Kind regards,
Loek

[rafalkowalewski1]

Hi Loek,

It turns out that we had a similar issue in our lab a while ago. The bad news is that we do not really know why it arose. One may think that the calibration file should not have an impact here - after all, all the values for spot width and height are interpolated during calibration (see the middle plot in the top row after calibration) so when fitting z for other files, it should not be a problem.

On the other hand, rerunning the calibration solved the issue for us! So, unfortunately, our only suggestion is that you should try to recalibrate your microscope until it works. Once your calibration plots do not have gaps in the data it should work.

Our hypothesis (not tested though) is that the fitted polynomial may have invalid sx/sy values at some depth and so the locs are dropped. In the function picasso.lib.ensure_sanity, localizations are also fitted for the return localization precision (only positive values are allowed), so our guess is that somehow fitting at these gaps returns negative sx/sy (lpx is approximately proportional to sx).

I hope this is of any help. I am aware this advice may not be ideal. However, I would be very happy to hear if you find the reason the error occured!

Bests,
Rafal

[rafalkowalewski1]

The bug fix reported in the email will be implemented in Picasso 0.7.0