-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Artifacts in match_template output after setting defocus search #508
Comments
Here are a few items that may help us troubleshoot: please grab another screen cap of the same output, but switch the tab to the image not the mip, and click the "show job details" please. Also, please show the corresponding search result without the defocus search. (with the mip as you have here, but adding the "show job details) And finally, please show a view through a slice or projection of your template, as well as letting us know the x,y,z dimensions. |
I'm having some permission issues when opening the .db file with the cisTEM program. The above screen captures were generated by another user; I can have them produce the requested screen captures later. I can, however, open the database file and jobs 286 and 287 complete normally (give reasonable outputs), but everything past job 288 (when defocus search was included) has similar artifacting issues. Sorry for the shoddily formatted info; it's the best I can do right now.
I've also tested the cli version of match_template both with and without the defocus search and the artifacting issues does not present itself there both before and after setting the defocus search. Since the cli interface of match_template is working fine so I suspect it has something to do with reading the SQL db. |
Also this is using CiSTEM version |
Sorry for the lag there, I've had some personal things that required most of my spare brain space the last couple of weeks. Thank you for the extra information about the database @mgiammar. All the values look fine, but your observation that the CLI works fine and that the bug persists in the GUI after turning off the defocus sweep is very interesting. Thanks also for the specific version, that will help with further trouble shooting. So there are two things to discuss. The observed bug, and the nature of your project. I'll separate these two. Bug discussion We have a few good symptoms to look at, so I think the next step will to be reproducibility on our end. @rezaparaanczi Please put together a compressed folder with the following that you can share via your preferred method.
Sample may not be suitable for current implementation of TM in cisTEM Clatherin! Let me start with a bit of a disclaimer: Putting this apparent bug aside for a moment, you are working with a sample that is really going to test the limits of the current implementation of template matching. 1 - the odd shape produces an orientation-dependent distribution of SNR values, which violates the underlying statistical hypothesis that comparing one noisy image to each of N independent orientations produces a distribution of SNR values similar to comparing one orientation to N independent noisy images. Finding a good solution to this is an active area of research, and I think it would be worth asking @timothygrant80 for a brief comment on whether he thinks your project is feasible currently. 2 - adding in a target environment that has dramatic differences in the local image variance further violates our current assumption that the noise in the image is stationary. We currently deal with this via "flat-fielding" the pixel wise mean and variance after the search. The problem is that with something shaped like clatherin, the statistical images we record for this process only tell us the first two moments of the underlying distribution. This is fine for a normal distribution, but for the triskelion shape, the pixel-wise distribution of SNR values is decided not Gaussian. |
@twagner9 I am not sure this has to do with the defocus sweep or if that is just a coincidence and something else is broken in the DB. Can you have a read over this issue and see if it is something you would be interested in taking a look at once the repro data are shared? |
@bHimes Yes, I'd be happy have a look at this once the sample data is available. |
Hi Ben,
Thank you so much for the feedback. I have shared a sample micrograph here
with a template of mtorc1. Hope this helps.
Best regards,
Reza
debugging.zip
<https://drive.google.com/file/d/1vrXkdowPmxpinEOVoXxqX0KTQ_s23dWo/view?usp=drive_web>
…On Sat, Jun 29, 2024 at 4:47 AM B.A.Himes ***@***.***> wrote:
Sorry for the lag there, I've had some personal things that required most
of my spare brain space the last couple of weeks.
Thank you for the extra information about the database @mgiammar
<https://github.com/mgiammar>. All the values look fine, but your
observation that the CLI works fine *and* that the bug persists in the
GUI after turning off the defocus sweep is very interesting. Thanks also
for the specific version, that will help with further trouble shooting.
So there are two things to discuss. The observed bug, and the nature of
your project. I'll separate these two.
*Bug discussion*
We have a few good symptoms to look at, so I think the next step will to
be reproducibility on our end. @rezaparaanczi
<https://github.com/rezaparaanczi> Please put together a compressed
folder with the following that you can share via your preferred method.
- The template used when encountering the error
- One micrograph where the error occured
- A text file with the imaging parameters needed to create a new
project (imaging parameters)
*Sample may not be suitable for current implementation of TM in cisTEM*
Clatherin! Let me start with a bit of a disclaimer: Putting this apparent
bug aside for a moment, you are working with a sample that is really going
to test the limits of the current implementation of template matching.
1 - the odd shape produces an orientation-dependent distribution of SNR
values, which violates the underlying statistical hypothesis that comparing
one noisy image to each of *N* independent orientations produces a
distribution of SNR values similar to comparing *one orientation* to *N*
independent noisy images.
Finding a good solution to this is an active area of research, and I think
it would be worth asking @timothygrant80
<https://github.com/timothygrant80> for a brief comment on whether he
thinks your project is feasible currently.
2 - adding in a target environment that has dramatic differences in the
local image variance further violates our current assumption that the noise
in the image is stationary. We currently deal with this via "flat-fielding"
the pixel wise mean and variance *after* the search. The problem is that
with something shaped like clatherin, the statistical images we record for
this process only tell us the first two moments of the underlying
distribution. This is fine for a *normal* distribution, but for the
triskelion shape, the pixel-wise distribution of SNR values is decided not
Gaussian.
—
Reply to this email directly, view it on GitHub
<#508 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BBR5VDAAEKQJ5X3QWO67WZ3ZJ2NF5AVCNFSM6AAAAABI5INGWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJYGEZDGMJSG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
After setting a defocus search range in the match_template program the resulting outputs show some odd line-like artifacting for the locations of identified particles. Here are what the results look like from the GUI
What's odd is the template match runs fine before setting a defocus search range, but once a defocus search range has been set for a template matching run this artifacting behavior persists for all future runs. Particle symmetry does not seem to have any affect on the outputs.
The unscaled MIP, correlation mean and variance outputs along with the best orientation angles show similar issues so I'm suspecting that the issue lies somewhere in the cross correlation step.
The text was updated successfully, but these errors were encountered: