What's the best way to process 2D acquisitions?

[cleterrier]

I'm trying to use decode for 2D SMLM acquisitions (no astigmatic lens, regular PSF). I acquired the corresponding PSF, fitted it with splines in SMAP, and trained DECODE on it - so it will actually try to fit the blinking events in 3D, even if it will be non ideal with a regular PSF (problem of non-determination between +Z and -Z due to symmetry, and uncertainty is worst around Z=0). I plan to only use the XY info anyway (although I'm curious to see how it will perform for Z).

Is that the best way? Or are there specific options to train DECODE and subsequently fit only for X and Y coordinates that would be faster/more precise?

[ASpeiser]

Hey,
sorry for the wait, we can't figure out how to get notifications for new topics posted here ...
About your question, there is a parameter setting for that scenario: param.HyperParameter.disabled_attributes = 3
should disable learning of Z. More precisely it will simply set the target to 0, so the network will learn to just output zeros everywhere.
I'm not sure that this has been tested much. It's not impossible that this does funky things with the loss (because you can get really high likelihoods). So maybe you give it ago and report back? If it behaves weirdly we can just set the sigma to a constant value.

[Haydnspass]

So far from my experience, it should work well. Possibly lower the lr if strange things happen (as always).

[cleterrier]

Will try and report if I have problems!

[cleterrier]

What I have done following your suggestion is to change the training yaml file as such:
disabled_attributes: 3 on line 60
And launching a training again.

The TensorBoard monitoring looked kinda strange at the beginning, with zero localizations on the inferred image, yet good values for the lateral RMSE on the graphs. Now it's getting better with some inferred localizations. It seems to be inferring only very bright and isolated emitters so far (50 epochs), compared to a reference 3D training I'll see what happens with more training, hopefully it will pick up fainter blinking events.

@ASpeiser @Haydnspass (tagging you so that you hopefully get a notification)

[cleterrier]

I have made the training available via TensorBoard.dev upload, so you can follow it live:
https://tensorboard.dev/experiment/0Xol8m6rT9GLPv1Js5SWyw/#scalars&_smoothingWeight=0

orange trace is a reference 3D training with astigmatic PSF
blue trace is a 3D training with the 2D PSF (no astigmatic lens)
red trace is the currently running 2D training with the 2D PSF (with the disabled_attributes: 3 in the yaml file)

Sadly the "images" tab is not available via upload.
Let me know if you think the current training will not result in a usable model?

[Haydnspass]

Indeed I got a notification :P
Yeah, the issue that you are seeing here is that your evaluation tries to match ground truth and predictions in 3D (because this is the default).
This does not affect the training but only the evaluation in the tensorboard.
What you can do is to add

Evaluation:
  match_dims: 2

to your training param file then ground truth and predictions will be matched only based on their lateral distance.

[cleterrier]

So even with Evaluation match_dims: 2, it's still detecting with quite a low efficiency compared to 3D, After more than 800 epochs it looks like this (left is 3D typical, righht is 2D):

@Haydnspass @ASpeiser

[cleterrier]

Basically I want to have a dev version of DECODE in a separate environment, that has everything installed and allows me to test his branch

[ASpeiser]

That looks like a jupyter issue though. Does any of this help: https://stackoverflow.com/questions/57677481/jupyter-command-jupyter-lab-not-found ?

Does "jupyter notebook" start?

[cleterrier]

What I tried now that worked:

• clone the repository
• cd into the DECODE folder
• switch to the test branch
• conda env create -f environment.yaml after making two modifications in the environment.yaml file: 1) cudatoolkit activated 2) added jupyterlab
• this created a decode_dev environment with the code change you made
• work with this environment/install of DECODE

Training is now underway, I'll let you know how it goes

[cleterrier]

Looks like it worked well! Got a good broad training (in terms of background values, so that it is versatile). Now processing a couple of 2D experiments

[ASpeiser]

Nice! Thanks for the continued testing.

[cleterrier]

Just wanted to add: using the patched dev version is required for training, but also for fitting with the resulting model.

I tried to train in the patched version and use the model in a vanilla 0.10 install, but it started fitting aberrant Z position rather than assigning Z=0 to all events (like the patched version does)

[ASpeiser]

Yeah, that makes sense. In the patched version the weights that produce the z-output are not trained and therefore stay at their random initialization. The non-patched version then uses these random weights to generate z-outputs. But you could also just manually set them to zero in the resulting emitter set.

[Haydnspass]

By the way that is an important point.
Although it might work with different versions, one should always use the exact version the model was trained on. I'll add this to the docs.
Maybe I'll have a look to generate output models that are self-contained binaries.

[pranayj77]

To fit 2D data, do I still need to work with the dev branch, or are all these issues fixed in the main branch?

[liuliuliu1111]

Hello! I have read the discussion and tried to set param.HyperParameter.disabled_attributes = 3, but it didn't work well on the model(version decode_v0_10_1rc1), as pointed by cleterrier before that a lot of events were missed. So is there a solution to this version and I can fix it by myself? Thanks a lot!

[liuliuliu1111]

I changed learning rate to 0.00001 since I found in the default case and param.HyperParameter.disabled_attributes = 3, I got negtive loss.

[Haydnspass]

Dear @liuliuliu1111
Thanks for sharing. However, your negative loss value is not an indicator of failed training.
In fact, the lower the loss the better and this is expected by the design of our loss function.

Do you need any further help?

[liuliuliu1111]

Yeah, I found that the model with negative loss performed quite good. It's relieved to know that this result is expected. I would like to know where does this parameter disabled_attributes come into play. Thank you!