Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare terminology files in JSON #42

Closed
fedorov opened this issue Jul 13, 2016 · 6 comments
Closed

Prepare terminology files in JSON #42

fedorov opened this issue Jul 13, 2016 · 6 comments

Comments

@fedorov
Copy link
Member

fedorov commented Jul 13, 2016

It probably makes sense to use JSON for all tasks.

@fedorov
Copy link
Member Author

fedorov commented Jul 31, 2016

Communication to @dclunie on Jul 28

for dcmqi, I am looking to reuse and rethink a bit how anatomic region
and category/type codes are organized.

The motivation is to simplify the process of codes selection for the
user by defining application-specific "contexts" (for the lack of a
better term) that include codes/combinations of codes relevant in a
specific use case.

These "contexts" would be used in the web application for populating
SEG meta-information (http://qiicr.org/dcmqi/#/seg), and in the
re-designed UI of Slicer.

The approach would be to

  1. instead of having separate XML files for 1) AnatomicRegion and
    modifier, and 2) Segmentation category/type/modifier, put both into
    sections of a JSON file (this will also be consistent with the rest of
    dcmqi, where we use JSON for metadata communication).
  2. have one big all-inclusive "context" that mirrors codes from the
    standard (for now, by just merging the content of the files you
    provided for ClearCanvas):

https://github.com/fedorov/ClearCanvas-AimPlugin4.5/blob/master/Segmentation/Configuration/AnatomicRegionAndModifier.xml
https://github.com/fedorov/ClearCanvas-AimPlugin4.5/blob/master/Segmentation/Configuration/SegmentationCategoryTypeModifier.xml

  1. Have several (added on as-needed basis) "contexts" relevant in
    QIICR-prioritized applications:
    • "Iowa use case context" - H&N anatomic locations, tumor, reference
      region, ...
    • "BWH use case context" - prostate anatomy, needle, biopsy, tumor, ...

Questions for you:

  1. what do you say?
  2. for Slicer, we would ideally like to somehow link color
    representation with the codes. Do you have any idea how this could be
    fit in? Would we assign a color to a combination of category/type, or
    somehow tie in anatomic region to location, or have different colors
    for category/type and anatomic region, and support different "facets"
    of colorization?

@fedorov
Copy link
Member Author

fedorov commented Jul 31, 2016

Response from @dclunie Jul 31

This sounds fine ... I don't think it is any harder to
automate the production of one monster file as opposed
to several smaller files from the standard (to be able
to update it as the standard evolves), nor does it make
a difference whether it is in XML or JSON, since we aren't
using features (like namespaces) that are not available
in JSON (and one can write JSON out of XSLT by setting
the output method to plain text).

As for generic vs. subset-ted contexts for specific use
cases, I totally agree ... there is no point in burdening
the users with stuff they don't need (in pick lists, etc.),
and our ClearCanvas design was intended to allow for use-
case specific configurations to be loaded, and what I
provided for them was just an "example" ... the hard part
is figuring out how to automate the production of the
subset from the master when the source of the master
changes (e.g., when a code changes).

Same goes for the colors ... you might want to have
a separate source that specifies the mapping for each
code that gets consulted when building the monster
master JSON file that includes it inline (i.e., creates
the master JSON file by merging the tables in the DICOM
standard with the information from the color table).

There is also the question of whether you want to allow
for different choices of colors for different use cases.

If one is going to do such merging, the question of needing
yet another code as a "primary key" to use that is immune
to changes in the external code (e.g., SNOMED code) arises
(e.g., should your color map source by index by a local
primary key, or by the SNOMED or UMLS or whatever code
that is actually used).

David

PS. There was actually an interesting question about
standardization of colors for anatomy that came across
DICOM WG 11's list this week (attached).

@fedorov
Copy link
Member Author

fedorov commented Aug 2, 2016

Slicer terminology+colors for completeness: http://wiki.slicer.org/slicerWiki/index.php/Documentation/4.2/Extensions/Reporting

@dclunie
Copy link
Member

dclunie commented Aug 3, 2016

Hi Andrey

I would not mention CP 1258 (incorrectly typed as 1528), since this
is now part of the standard, and we don't want people using obsolete
text from CPs.

I would reference CIDs 7150 and 7151 in the standard instead perhaps:

http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_7150.html
http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_7151.html

(and if you want to describe the history, reference CP 1258 only in
that context and emphasize not to use it but to refer to the current
standard).

Also, you are using a non-standard representation of the code tuples,
with semi-colons rather than commas, and we normally quote the meaning
since it may contain spaces and punctuation, e.g., instead of:

(T-D000A;SRT;Anatomical Structure)

I suggest:

(T-D000A,SRT,"Anatomical Structure")

I have NOT checked all the entries in the LUT table to make sure
that they are consistent with the current standard ... do you
want me to do that? I can probably automate it, if I can first
figure out how to get the table out of the wiki ... I guess I
would have to write a wiki table parser to do that. A search
suggests there may already be tools (http://wikitables.geeksta.net/);
yes, it works:

http://wikitables.geeksta.net/url/?url=http%3A%2F%2Fwiki.slicer.org%2FslicerWiki%2Findex.php%3Ftitle%3DDocumentation%2F4.2%2FExtensions%2FReporting

David

PS. Is there a reason this information is in the 4.2 documentation
and the head of the page says that 4.5 is the most recent version?

On 8/2/16 4:48 PM, Andrey Fedorov wrote:

Slicer terminology+colors for completeness: http://wiki.slicer.org/slicerWiki/index.php/Documentation/4.2/Extensions/Reporting


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#42 (comment)

@fedorov
Copy link
Member Author

fedorov commented Aug 3, 2016

I would not mention CP 1258 (incorrectly typed as 1528), since this
is now part of the standard, and we don't want people using obsolete
text from CPs.

I would reference CIDs 7150 and 7151 in the standard instead perhaps:

http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_7150.html
http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_7151.html

(and if you want to describe the history, reference CP 1258 only in
that context and emphasize not to use it but to refer to the current
standard).

Done, I updated the page

Also, you are using a non-standard representation of the code tuples,
with semi-colons rather than commas, and we normally quote the meaning
since it may contain spaces and punctuation

The master spreadsheet is here: https://goo.gl/vuANnw, but I don't know how it is related to the wiki page, if either one was updated later. I think I had a conversion script from csv to wiki, but I am not sure where.

I have NOT checked all the entries in the LUT table to make sure
that they are consistent with the current standard ... do you
want me to do that?

No, I don't! I will reuse the version from the drive and see how it can be reconciled with the ClearCanvas terminology compendum, to put together colors.

PS. Is there a reason this information is in the 4.2 documentation
and the head of the page says that 4.5 is the most recent version?

This "table" migrated to the core Slicer application at some point. Slicer main application does not have versioned documentation of color tables at that level, or it is not maintained, not sure. Slicer documentation is imperfect, that is the reason I guess...

fedorov added a commit to fedorov/dcmqi that referenced this issue Aug 5, 2016
Approach implemented:

 - "SegmentationCategoryTypeContext" defines the list of category/type/modifier
 combinations
 - SegmentationCategoryTypeModifier.json is the result of converting
 SegmentationCategoryTypeModifier.xml, taken from "AIM on ClearCanvas", into JSON
 using the supplied xml2json.py script, with a small extra edit to account for
 conversion inconsistency
 - SlicerGenericAnatomy.csv is the result of importing first sheet of this
 document https://goo.gl/vuANnw in CSV, with small extra edits to account for missing
 quotes (also fixed in the master google doc)
 - SegmentationCategoryTypeModifierRGB.json is the result of adding RGB to the
 ClearCanvas JSON category/type by matching with CSV using the supplied
 mergeJSONCodesWithCSVColors.py script

In a similar fashion, more narrowly-defined contexts could be defined, with a
smaller number of codes tailored to a specific application.

This is work towards QIICR#42
@fedorov
Copy link
Member Author

fedorov commented Aug 5, 2016

@dclunie: After reconciling the "AIM on ClearCanvas" category/type/modifier compendum with the list of codes we put together for Slicer (at least, a version of it - https://goo.gl/vuANnw!), there are some codes that are present in the Slicer list, but not in the ClearCanvas list. Here is the complete list of 49 entries that were not matched (category/type/modifier code tuples):

(T-D000A,SRT,"Anatomical Structure") (T-62000,SRT,"Liver") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-26000,SRT,"Bronchus") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A6620,SRT,"Superior cerebellar peduncle") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-A6620,SRT,"Superior cerebellar peduncle") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-11150,SRT,"Sphenoid bone") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-11150,SRT,"Sphenoid bone") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-A5100,SRT,"Midbrain") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-D8600,SRT,"Wrist") (G-A101,SRT,"Left")
  1 ENH: initial "SegmentationCategoryTypeContext" added
(T-D000A,SRT,"Anatomical Structure") (T-11140,SRT,"Occipital bone") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-48081,SRT,"Systemic vein") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A8640,SRT,"Vagus nerve") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-D2500,SRT,"Hip") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-D2500,SRT,"Hip") (G-A100,SRT,"Right")
(T-D0080,SRT,"Body Substance") (T-C2000,SRT,"Blood") (,,)
(T-D0050,SRT,"Tissue") (T-A0096,SRT,"Gray matter") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-29000,SRT,"Pleura") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-C3000,SRT,"Spleen") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-11156,SRT,"Ethmoid bone") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-D0821,SRT,"Bone of upper limb") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-11156,SRT,"Ethmoid bone") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-13600,SRT,"Muscle of upper limb") (,,)
(R-42018,SRT,"Spatial and Relational Concept") (new,DCM,"Background") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-21342,SRT,"Vomer bone") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-11300,SRT,"Rib") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-48581,SRT,"Pulmonary vein") (,,)
(T-D0050,SRT,"Tissue") (T-A2020,SRT,"Cerebral gray matter") (,,)
(T-D0080,SRT,"Body Substance") (T-A1000,SRT,"Cerebrospinal fluid") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A0103,SRT,"Telencephalon") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-C6000,SRT,"Lymphatic system") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-88000,SRT,"Fallopian tube") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-D0678,SRT,"Elbow") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-30000,SRT,"Cardiovascular system") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-D0678,SRT,"Elbow") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-11170,SRT,"Maxilla") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-D8600,SRT,"Wrist") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-21000,SRT,"Nose") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-44000,SRT,"Pulmonary artery") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A5160,SRT,"Substantia nigra") (,,)
(T-D0050,SRT,"Tissue") (T-A0095,SRT,"White matter") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-12700,SRT,"Bone of lower limb") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-4105E,SRT,"Systemic artery") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A1740,SRT,"Third ventricle") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-A1740,SRT,"Third ventricle") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-B7000,SRT,"Parathyroid") (G-A100,SRT,"Right")
(T-D000A,SRT,"Anatomical Structure") (T-B7000,SRT,"Parathyroid") (G-A101,SRT,"Left")
(T-D000A,SRT,"Anatomical Structure") (T-14668,SRT,"Muscle of lower limb") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-A9630,SRT,"Sympathetic trunk") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-73000,SRT,"Ureter") (,,)
(T-D000A,SRT,"Anatomical Structure") (T-32100,SRT,"Atrium") (,,)

Looking at some of the discrepancies, some of them are minor (e.g., (T-A0095,SRT,"White matter") in the Slicer list vs ``(T-A2030,SRT,"Cerebral White Matter")` in the ClearCanvas list. Same goes for grey matter, "Elbow" vs "Elbow joint". In some cases, discrepancy is due that Slicer list does not apply laterality to some concepts (Maxilla), or the other way around (Parathyroid).

Some discrepancies are more substantial. For example, "Blood" is in the "Body substance" category in the Slicer list, but in the "Tissue" category in the ClearCanvas list. Considering that Blood is in CID 7166. Common Tissue Segmentation Types, I assume it is the Slicer list that is not correct.

I do not think it is urgent to reconcile the two tables. The list of codes used by Slicer will be replaced with the JSON-based segmentation context lists provided by dcmqi. So I hope you don't spend too much time on this. I can spend a bit more time later to reconcile the two, and let you know about any detailed substantial differences.

For now, we have a ClearCanvas category/type/modifier list with RGB colors assigned from the Slicer list, so we can follow the same pattern of "segmentation category type context" for QIICR use cases, and also use these JSON files to improve both the dcmqi/seg web application, and support of terminology+colors in Slicer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants