New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and select: attaching nodule centroid marker to current slice #250

Merged
merged 4 commits into from Nov 29, 2017

Conversation

Projects
None yet
3 participants
@Serhiy-Shekhovtsov
Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 25, 2017

Implemented slice switching for candidate list, and a way for user to see when slice differs and attach the marker to current slice.

Description

Following features has been implemented:

  • switching the viewer to candidate's slice when selecting a candidate
  • showing to user that marker is on an other slice
  • clicking on a marker attaches it to current slice

On back-end I have added a method for loading candidates with cases and series. Series object will also have list of DICOM files in the folder saved in uri property.

Reference to official issue

It's an improvement of #148

Screenshot:

detect-and-select-z
full size image

CLA

  • I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well

@Serhiy-Shekhovtsov Serhiy-Shekhovtsov force-pushed the Serhiy-Shekhovtsov:features/detect-and-select-z branch 2 times, most recently from 1ebaad5 to 6ee62df Nov 25, 2017

series = candidate['case']['series']

if 'files' not in series:
series['files'] = glob.glob1(series['uri'], '*.dcm')

This comment has been minimized.

@lamby

lamby Nov 26, 2017

Contributor

Can you document why glob1 and not regular glob?

This comment has been minimized.

@Serhiy-Shekhovtsov

Serhiy-Shekhovtsov Nov 26, 2017

Contributor

I have put a comment. Btw, this entire piece should be moved from here. We should discover and save list of files only once - when creating a case. So as soon as #145 is done(PR #233 is pending), we will move it.

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 26, 2017

Ran this on aws and am getting a few errors:

...
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py", line 68, in view
interface_1   |     return self.dispatch(request, *args, **kwargs)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/rest_framework/views.py", line 489, in dispatch
interface_1   |     response = self.handle_exception(exc)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/rest_framework/views.py", line 449, in handle_exception
interface_1   |     self.raise_uncaught_exception(exc)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/rest_framework/views.py", line 486, in dispatch
interface_1   |     response = handler(request, *args, **kwargs)
interface_1   |   File "/app/backend/api/views.py", line 70, in get
interface_1   |     ds = dicom.read_file(path, force=True)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/dicom/filereader.py", line 614, in read_file
interface_1   |     force=force)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/dicom/filereader.py", line 553, in read_partial
interface_1   |     stop_when=stop_when, defer_size=defer_size)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/dicom/filereader.py", line 305, in read_dataset
interface_1   |     raw_data_element = next(de_gen)
interface_1   |   File "/usr/local/lib/python3.6/dist-packages/dicom/filereader.py", line 219, in data_element_generator
interface_1   |     value = fp_read(length)
interface_1   | MemoryError
...
vue_1         | [HPM] Proxy created: /  ->  http://interface:8000
vue_1         | > Starting dev server...
vue_1         |  ERROR  Failed to compile with 1 errors04:30:22
vue_1         | 
vue_1         |  error  in ./src/components/open-image/OpenDICOM.vue
vue_1         | 
vue_1         | 
vue_1         |   ✘  http://eslint.org/docs/rules/indent  Expected indentation of 8 spaces but found 6  
vue_1         |   src/components/open-image/OpenDICOM.vue:76:9
vue_1         |           this.stack.currentImageIdIndex = this.view.paths.indexOf(this.view.state)
vue_1         |            ^
vue_1         | 
vue_1         |   ✘  http://eslint.org/docs/rules/indent  Expected indentation of 8 spaces but found 6  
vue_1         |   src/components/open-image/OpenDICOM.vue:77:9
vue_1         |           if (this.stack.currentImageIdIndex < 0) this.stack.currentImageIdIndex = 0
vue_1         |            ^
vue_1         | 
vue_1         | 
vue_1         | ✘ 2 problems (2 errors, 0 warnings)
vue_1         | 
vue_1         | 
vue_1         | Errors:
vue_1         |   2  http://eslint.org/docs/rules/indent
vue_1         | 
vue_1         |  @ ./~/babel-loader/lib!./~/vue-loader/lib/selector.js?type=script&index=0&bustCache!./src/components/detect-and-select/CandidateList.vue 49:0-48
vue_1         |  @ ./src/components/detect-and-select/CandidateList.vue
vue_1         |  @ ./~/babel-loader/lib!./~/vue-loader/lib/selector.js?type=script&index=0&bustCache!./src/views/DetectAndSelect.vue
vue_1         |  @ ./src/views/DetectAndSelect.vue
vue_1         |  @ ./src/routes.js
vue_1         |  @ ./src/main.js
vue_1         |  @ multi ./build/dev-client ./src/main.js
vue_1         | 
vue_1         | > Listening at http://0.0.0.0:8080
...

And navigating to port 8080 returns a 404 error with Cannot GET /.

@Serhiy-Shekhovtsov Serhiy-Shekhovtsov force-pushed the Serhiy-Shekhovtsov:features/detect-and-select-z branch 3 times, most recently from 79235b2 to b97dd9c Nov 26, 2017

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 26, 2017

@reubano I have fixed mentioned issues.

@lamby

This comment has been minimized.

Copy link
Contributor

lamby commented Nov 27, 2017

Build seems to be failing though? :)

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 27, 2017

I also noticed port 8080 was still erroring. I ran git bisect and it seems the failure is from a previous commit. Gimme a sec to find it.

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

So according to git bisect, this commit is the first bad one.

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

So this is the error from travis

=================================== FAILURES ===================================
____________________________ test_lung_segmentation ____________________________
self = (0002, 0001), other = (65534, 57357)
    def __eq__(self, other):
        # Check if comparing with another Tag object; if not, create a temp one
        if not isinstance(other, BaseTag):
            try:
>               other = Tag(other)
/usr/local/lib/python3.6/dist-packages/dicom/tag.py:62: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
arg = (65534, 57357), arg2 = None
    def Tag(arg, arg2=None):
        """General function for creating a Tag in any of the standard forms:
        e.g.  Tag(0x00100010), Tag(0x10,0x10), Tag((0x10, 0x10))
        """
        if arg2 is not None:
            arg = (arg, arg2)  # act as if was passed a single tuple
>       if isinstance(arg, (tuple, list)):
/usr/local/lib/python3.6/dist-packages/dicom/tag.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
signum = 14, frame = <frame object at 0x7faadaf79cf8>
    def _timeout(signum, frame):
>       raise TimeoutExit("Runner timeout is reached, runner is terminating.")
E       app.src.tests.conftest.TimeoutExit: Runner timeout is reached, runner is terminating.
src/tests/conftest.py:96: TimeoutExit
During handling of the above exception, another exception occurred:
dicom_paths = ['/images_full/LIDC-IDRI-0001/1.3.6.1.4.1.14519.5.2.1.6279.6001.298806137288633453246975630178/1.3.6.1.4.1.14519.5.2.1...14519.5.2.1.6279.6001.490157381160200744295382098329/1.3.6.1.4.1.14519.5.2.1.6279.6001.619372068417051974713149104919']
    @pytest.mark.stop_timeout
    def test_lung_segmentation(dicom_paths):
        """Test whether the annotations of the LIDC images are inside the segmented lung masks.
        Iterate over all local LIDC images, fetch the annotations, compute their positions within the masks and check that
        at this point the lung masks are set to 255."""
    
        for path in dicom_paths:
            min_z, max_z = get_z_range(path)
            directories = path.split('/')
            lidc_id = directories[2]
            patient_id = directories[-1]
>           original, mask = save_lung_segments(path, patient_id)
src/tests/test_segmentation.py:49: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/preprocess/lung_segmentation.py:61: in save_lung_segments
    slices = load_patient(dicom_path)
src/preprocess/lung_segmentation.py:95: in load_patient
    dicom_slice = dicom.read_file(os.path.join(src_dir, s))
/usr/local/lib/python3.6/dist-packages/dicom/filereader.py:614: in read_file
    force=force)
/usr/local/lib/python3.6/dist-packages/dicom/filereader.py:521: in read_partial
    file_meta_dataset = _read_file_meta_info(fileobj)
/usr/local/lib/python3.6/dist-packages/dicom/filereader.py:447: in _read_file_meta_info
    is_little_endian=True, stop_when=not_group2)
/usr/local/lib/python3.6/dist-packages/dicom/filereader.py:309: in read_dataset
    if tag == (0xFFFE, 0xE00D):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = (0002, 0001), other = (65534, 57357)
    def __eq__(self, other):
        # Check if comparing with another Tag object; if not, create a temp one
        if not isinstance(other, BaseTag):
            try:
                other = Tag(other)
            except:
>               raise TypeError("Cannot compare Tag with non-Tag item")
E               TypeError: Cannot compare Tag with non-Tag item
/usr/local/lib/python3.6/dist-packages/dicom/tag.py:64: TypeError
----------------------------- Captured stderr call -----------------------------
ERROR:root:69.xml is no valid DICOM
=============================== warnings summary ===============================
src/tests/test_endpoints.py::test_identify
  /app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
    truncate=2.0) < intensity_th
src/tests/test_identification.py::test_identify_nodules_001
  /app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
    truncate=2.0) < intensity_th
src/tests/test_identification.py::test_identify_nodules_003
  /app/src/tests/../../src/preprocess/extract_lungs.py:34: RuntimeWarning: invalid value encountered in less
    truncate=2.0) < intensity_th
-- Docs: http://doc.pytest.org/en/latest/warnings.html
========= 1 failed, 47 passed, 4 xfailed, 3 warnings in 820.90 seconds =========
The command "sh tests/test_docker.sh" exited with 1.
Done. Your build exited with 1.

I also rebuilt it, and port 8080 loads now, but clicking 'Detect and Select' doesn't do anything.

@Serhiy-Shekhovtsov Serhiy-Shekhovtsov force-pushed the Serhiy-Shekhovtsov:features/detect-and-select-z branch from b97dd9c to 3fd27e9 Nov 28, 2017

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 28, 2017

@reubano, @lamby PR has been fixed by rebasing it of top of last commit that actually fixed the issue.

but clicking 'Detect and Select' doesn't do anything.

This is because of the Router Guard(#239) implementation :) You should click the green bar to make it let you access the next screen:
example

@Serhiy-Shekhovtsov Serhiy-Shekhovtsov referenced this pull request Nov 28, 2017

Merged

Detect and select: mark\dismiss candidates #254

1 of 1 task complete
@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

What i see now...

screen shot 2017-11-28 at 14 41 48

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 28, 2017

This is because of bad dummy data. Unfortunately #145 is still not closed, so the proper way to start the case doesn't work.

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

Hmmm, how did you get your screenshots then? The image shows up for me on the master branch.

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 28, 2017

I had the Start case feature implemented and working on my end. So I had the data with proper series.uri values. But we had conflicting Pull Requests for the same feature and I closed mine one if favor of an other.

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 28, 2017

@reubano this will give you correct data:

    from backend.cases.factories import *
    from backend.cases.models import *
    from backend.images.models import *

    # drop existing case(s) with candidates and nodules
    Case.objects.all().delete()

    # drop imported images
    ImageSeries.objects.all().delete()

    # import a new image and start a new case
    new_image, created = ImageSeries.get_or_create('/images/LIDC-IDRI-0003/1.3.6.1.4.1.14519.5.2.1.6279.6001.101370605276577556143013894866/1.3.6.1.4.1.14519.5.2.1.6279.6001.170706757615202213033480003264')
    new_case = CaseFactory(series=new_image)

    # dummy candidates
    candidates = CandidateFactory.create_batch(5, case=new_case)
    NoduleFactory(candidate=candidates[0], case=new_case)
    NoduleFactory(candidate=candidates[1], case=new_case)
    NoduleFactory(candidate=candidates[4], case=new_case)
@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

Thanks, that worked! But the image only appears after moving the slider to a new slice.

@Serhiy-Shekhovtsov

This comment has been minimized.

Copy link
Contributor

Serhiy-Shekhovtsov commented Nov 28, 2017

That's because dummy candidates are located on random and often missing slices. Opening a candidate will try to open that slice.

@reubano

This comment has been minimized.

Copy link
Contributor

reubano commented Nov 28, 2017

Gotcha. Nice work! Well, LGTM.

@reubano reubano merged commit 120e5df into drivendataorg:master Nov 29, 2017

2 checks passed

concept-to-clinic/cla @Serhiy-Shekhovtsov has signed the CLA.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment