added imaging tutorial #1534

ThomasWollmann · 2019-08-02T13:23:35Z

No description provided.

shiltemann

yay for an imaging topic, awesome!

If you could think of some question boxes for learners maybe that could be great? Just to get them to think about the outputs and what they are doing, and to check if they understood the most important points.

and could you add a workflow file? :)

topics/imaging/tutorials/imaging-introduction/tutorial.md

shiltemann · 2019-08-02T14:03:00Z

topics/imaging/tutorials/imaging-introduction/tutorial.md

+
+# Getting data
+
+The dataset required for this tutorial contains a screen of DAPI stained HeLa nuclei ([more information](https://zenodo.org/record/3358874#.XUQ9oi1XZUM)). We will use a sample image from this dataset for training basic imaging skills in Galaxy.


Could you give a bit more of an introduction/motivation? what are these images? what is the research/biological question we are trying to answer in this tutorial?

This tutorial is just introductory to image handling there is no research/biological question. "We will use a sample image from this dataset for training basic imaging skills in Galaxy."

it doesn't have to be much, even if it is just "we have an image of cells and want to detect the nuclei and segment the image"

just a thought, is it possible to add a step at the end that counts the number of cells? because that would be a neat little result maybe (but no worries if not possible)

topics/imaging/tutorials/imaging-introduction/tutorial.md

shiltemann · 2019-08-02T15:32:35Z

I just ran through the tutorial, it was fun, thanks!

I would maybe add some of the png image outputs to the manual, they are cool, and if people don't have time to finish or if they mess up, then they can at least see the output they were supposed to get, what do you think?
The input dataset is large (500Mb) and we only use one image (2.6Mb) from it, could we just upload the single file to zenodo? or if unzipping is something you want to teach them, maybe just create a zip file with only that image and possibly one or two others? it will save some time on import and some disk space on the server
Please still add a workflow file btw (since I just performed the tutorial I could extract it from my history too if you prefer)

again, very cool, thanks a lot!

ThomasWollmann · 2019-08-02T15:55:37Z

I just ran through the tutorial, it was fun, thanks!

I would maybe add some of the png image outputs to the manual, they are cool, and if people don't have time to finish or if they mess up, then they can at least see the output they were supposed to get, what do you think?

Yes this would be nice! Since, you already have the result images you can maybe just upload them :)

The input dataset is large (500Mb) and we only use one image (2.6Mb) from it, could we just upload the single file to zenodo? or if unzipping is something you want to teach them, maybe just create a zip file with only that image and possibly one or two others? it will save some time on import and some disk space on the server

Soon, I will add a second tutorial, which is using the whole dataset and should be teached along with this tutorial. Therefore, I think we can just keep the large dataset. Also makes sense to teach people from the beginning that imaging datasets are often not just single files.

Please still add a workflow file btw (since I just performed the tutorial I could extract it from my history too if you prefer)

Would be nice if you can add it, since you have it already. Thank you in advance!

again, very cool, thanks a lot!

You're welcome! Also checkout https://www.denbi.de/training/73-microscopy-image-analysis-course were we will teach a whole course on microscopy imaging and hopefully will use this workflow.

shiltemann · 2019-08-02T16:03:55Z

@ThomasWollmann ok, no problem, I will add the images and workflow on Monday :)

and cool about the workshop! have a great weekend!

shiltemann · 2019-08-05T11:12:01Z

Hi @ThomasWollmann I added the images and workflow, could you please check if it looks ok to you?

In the final image, I get a yellow label of 65535 on the image, is that meant to happen or did I do something wrong somewhere? and if not, maybe explain what this label signifies?

shiltemann · 2019-08-05T11:13:22Z

topics/imaging/tutorials/imaging-introduction/tutorial.md

+>    - {% icon param-file %} *"Input Image"*: `input_smoothed_normalized` file (output of **Histogram equalization** {% icon tool %})
+>    - *"Output data type"*: `PNG`
+> 6. Rename {% icon galaxy-pencil %} the generated file to `viz_smoothed_normalized`
+> 7. Click on the {% icon galaxy-eye %} (eye) icon next to the file name, to look at the file content and compare the result with `viz_normalized`.


here you ask them to compare the two images, could you maybe write a sentence about what they should observe (maybe under the image below)?

ThomasWollmann · 2019-08-05T11:36:21Z

Hi @ThomasWollmann I added the images and workflow, could you please check if it looks ok to you?

In the final image, I get a yellow label of 65535 on the image, is that meant to happen or did I do something wrong somewhere? and if not, maybe explain what this label signifies?

Yes, I missed a step. I'll add it to the tutorial and then you can try again ;)

ThomasWollmann · 2019-08-05T15:39:58Z

@shiltemann I changed the description. Can you please try it again and update the results?

shiltemann · 2019-08-06T08:34:29Z

topics/imaging/tutorials/imaging-introduction/tutorial.md

+> 3. **Binary 2 Label** {% icon tool %} with the following parameters to segment the image:
+>    - {% icon param-file %} *"Binary Image File"*: `input_segmented` file (output of **Auto Threshold** {% icon tool %})
+> 4. Rename {% icon galaxy-pencil %} the generated file to `input_segmented_labeled`
+> 5. **Convert image** {% icon tool %} with the following parameters to convert the image to PNG:


@ThomasWollmann was just testing this to extract the updated workflow, but this image now looks completely black, is that expected?

@shiltemann Guess this is the desired output. However, it is not nicely visible due to the low label ids. Try running histogram normalization before converting it. If this works, we can add it to the tutorial.

@ThomasWollmann ah yes, that's nicer :) I was thinking, what if we also do a Count objects step at the end, that would be a nice little result/objective maybe "here is an image, how many cells does it contain"? what do you think?

@shiltemann Nice idea. Feel free to add the result, since you already have the files in history. I'll add the text except the number.

ThomasWollmann · 2019-08-06T09:27:13Z

@shiltemann I added the extra step and some questions regarding this issue to the tutorial. Do you think 30 minutes is too optimistic for this tutorial?

ThomasWollmann · 2019-08-08T11:12:45Z

@shiltemann I changed the dataset and the description. I think the second tutorial is now more useful for practitioners. Please take a look!

ThomasWollmann · 2019-08-08T11:21:53Z

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

bgruening · 2019-08-08T11:38:58Z

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

Did you had issues? I could run the other completely. I would like to keep the old datasets and workflow for testing. I think its a cool example where we can benchmark the system.

ThomasWollmann · 2019-08-08T11:43:11Z

@bgruening Now, the splitted dataset is small and only two subsets are required for the tutorials. This should solve the issues for EU?

Did you had issues? I could run the other completely. I would like to keep the old datasets and workflow for testing. I think its a cool example where we can benchmark the system.

I just took time. The workflow is still the same. Now I splitted the dataset into the different treatments and in the course you can actually compare phenotypes visually or apply machine learning on the extracted features.

The full dataset is still on Zenodo. I think it is a good example for preparing Galaxy for large scale imaging datasets. Keep in mind that this dataset is still very small ;)

shiltemann · 2019-08-09T13:43:49Z

thanks a lot @ThomasWollmann :) The second tutorial looks good, I like that they also learn about workflows a bit at the same time :)

I reworked the last hands_on section of the first tutorial a bit, because using the normalized image resulted in higher IDs that overlapped in the final image and didnt look as nice

so please check that it all still looks ok to you :)

ThomasWollmann · 2019-08-09T14:47:32Z

thanks a lot @ThomasWollmann :) The second tutorial looks good, I like that they also learn about workflows a bit at the same time :)

I reworked the last hands_on section of the first tutorial a bit, because using the normalized image resulted in higher IDs that overlapped in the final image and didnt look as nice

so please check that it all still looks ok to you :)

Hmm since the image contains many objects, we probably should not display the ids. In the overlay tool you can set "Plot Labels" to "No". I think this looks much more useful. @shiltemann What do you think?

@shiltemann Thank you for putting so much effort in improving the quality of the tutorials!

shiltemann · 2019-08-09T15:01:07Z

oh, that was just how it looked before, to explain why I changed it back. ..without the histogram equalization step it looks quite ok now:

So I just reverted to your original steps, and made the histogram equalization step an exercise to help explain the completely black looking image they get, but use the original image for the segmentation mask overlay. What do you think?

ThomasWollmann · 2019-08-09T15:34:46Z

oh, that was just how it looked before, to explain why I changed it back. ..without the histogram equalization step it looks quite ok now:

So I just reverted to your original steps, and made the histogram equalization step an exercise to help explain the completely black looking image they get, but use the original image for the segmentation mask overlay. What do you think?

Sounds good! Now everything looks fine :)

ThomasWollmann · 2019-08-13T07:24:05Z

@shiltemann @bgruening still something missing for this tutorial?

bgruening · 2019-08-13T08:19:36Z

topics/imaging/tutorials/hela-screen-analysis/tutorial.md

@@ -95,7 +101,7 @@ The dataset required for this tutorial contains a screen of DAPI stained HeLa nu
 > 9. Rename {% icon galaxy-pencil %} the collection to `PLK1`
 > 9. Upload {% icon galaxy-upload %} the following segmentation filter rules as a new pasted file (format: `tabular`):
 >    ```
->    area	eccentricity
+>    	area	eccentricity


that looks wrong here. the first one

Why? The file has three columns. The first column has no heading.

You are correct. I have seen two ... but it seems to be just one.

👍 from me.

shiltemann · 2019-08-13T09:44:25Z

@shiltemann @bgruening still something missing for this tutorial?

Hi @ThomasWollmann oh, you mentioned you were planning to add more biological motivation so I thought you were still working on this

Could you just add some key_points to the tutorial metadata? just the main conclusions/take home messages you want people to remember from the tutorials.

Also: do we need to cite the person who created the dataset somehow? was it part of a publication or project we can refer to?

ThomasWollmann · 2019-08-13T11:15:17Z

@shiltemann The data was created by Manuel Gunkel as part of de.NBI Systematic Phenotyping. I do not think that we need additional citations, since we have the zenodo record.

ThomasWollmann · 2019-08-13T11:43:08Z

@shiltemann Now, I added some more motivation and key_points ;)

shiltemann · 2019-08-13T11:59:04Z

@ThomasWollmann thanks! This looks great!

ThomasWollmann · 2019-08-13T11:59:55Z

Wow, @shiltemann awesome work. Thank you for the support!

shiltemann · 2019-08-13T12:51:31Z

🎉

added imaging tutorial

92ca485

ThomasWollmann requested review from bebatut, hexylena and shiltemann as code owners August 2, 2019 13:23

added imaging tutorial

9fa443e

shiltemann reviewed Aug 2, 2019

View reviewed changes

Thomas Wollmann added 3 commits August 2, 2019 16:38

added imaging tutorial

0eedcc7

Merge remote-tracking branch 'origin/master'

e6add6e

removed unused file

a43f7cf

shiltemann added new contributor(s) new topic new tutorial imaging labels Aug 2, 2019

Thomas Wollmann and others added 4 commits August 5, 2019 10:17

updated src file

973f4de

add workflow

2e42bf8

add images

653a97c

add images to tutorial and small formatting tweaks

543d3ec

shiltemann reviewed Aug 5, 2019

View reviewed changes

shiltemann and others added 2 commits August 5, 2019 14:30

add alt text

ad9ed44

included comments

db7ef7c

shiltemann reviewed Aug 6, 2019

View reviewed changes

added missing steps

93a63b8

spelling

54bc2b6

reduced dataset size and made hela screen tutorial more useful

effc890

shiltemann added 4 commits August 9, 2019 12:32

update images, workflow, and final hands on box

5eca26d

fix numbering

3ab388c

update answer

33ce896

formatting tweaks

4418dc2

add alt text

b44c80a

fixed error + added recommended follow up training

5947ec7

bgruening reviewed Aug 13, 2019

View reviewed changes

shiltemann added 2 commits August 13, 2019 11:52

update formatting

6465792

update formatting

a9f85b7

Thomas Wollmann and others added 3 commits August 13, 2019 13:43

Added motivation and key_points

7213140

spelling

2310da7

minor tweaks

5cabacf

shiltemann approved these changes Aug 13, 2019

View reviewed changes

shiltemann merged commit f150522 into galaxyproject:master Aug 13, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added imaging tutorial #1534

added imaging tutorial #1534

ThomasWollmann commented Aug 2, 2019

shiltemann left a comment

shiltemann Aug 2, 2019

ThomasWollmann Aug 2, 2019 •

edited

Loading

shiltemann Aug 2, 2019

shiltemann commented Aug 2, 2019

ThomasWollmann commented Aug 2, 2019 •

edited

Loading

shiltemann commented Aug 2, 2019

shiltemann commented Aug 5, 2019

shiltemann Aug 5, 2019

ThomasWollmann commented Aug 5, 2019

ThomasWollmann commented Aug 5, 2019

shiltemann Aug 6, 2019

ThomasWollmann Aug 6, 2019 •

edited

Loading

shiltemann Aug 6, 2019

ThomasWollmann Aug 6, 2019

ThomasWollmann commented Aug 6, 2019

ThomasWollmann commented Aug 8, 2019

ThomasWollmann commented Aug 8, 2019

bgruening commented Aug 8, 2019

ThomasWollmann commented Aug 8, 2019 •

edited

Loading

shiltemann commented Aug 9, 2019

ThomasWollmann commented Aug 9, 2019

shiltemann commented Aug 9, 2019 •

edited

Loading

ThomasWollmann commented Aug 9, 2019

ThomasWollmann commented Aug 13, 2019

bgruening Aug 13, 2019

ThomasWollmann Aug 13, 2019

bgruening Aug 13, 2019

shiltemann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

shiltemann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

shiltemann commented Aug 13, 2019


		# Getting data

		The dataset required for this tutorial contains a screen of DAPI stained HeLa nuclei ([more information](https://zenodo.org/record/3358874#.XUQ9oi1XZUM)). We will use a sample image from this dataset for training basic imaging skills in Galaxy.

added imaging tutorial #1534

added imaging tutorial #1534

Conversation

ThomasWollmann commented Aug 2, 2019

shiltemann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ThomasWollmann Aug 2, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shiltemann commented Aug 2, 2019

ThomasWollmann commented Aug 2, 2019 • edited Loading

shiltemann commented Aug 2, 2019

shiltemann commented Aug 5, 2019

Choose a reason for hiding this comment

ThomasWollmann commented Aug 5, 2019

ThomasWollmann commented Aug 5, 2019

Choose a reason for hiding this comment

ThomasWollmann Aug 6, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ThomasWollmann commented Aug 6, 2019

ThomasWollmann commented Aug 8, 2019

ThomasWollmann commented Aug 8, 2019

bgruening commented Aug 8, 2019

ThomasWollmann commented Aug 8, 2019 • edited Loading

shiltemann commented Aug 9, 2019

ThomasWollmann commented Aug 9, 2019

shiltemann commented Aug 9, 2019 • edited Loading

ThomasWollmann commented Aug 9, 2019

ThomasWollmann commented Aug 13, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shiltemann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

shiltemann commented Aug 13, 2019

ThomasWollmann commented Aug 13, 2019

shiltemann commented Aug 13, 2019

ThomasWollmann Aug 2, 2019 •

edited

Loading

ThomasWollmann commented Aug 2, 2019 •

edited

Loading

ThomasWollmann Aug 6, 2019 •

edited

Loading

ThomasWollmann commented Aug 8, 2019 •

edited

Loading

shiltemann commented Aug 9, 2019 •

edited

Loading