Pattern Discovery / Categorize #167
Conversation
|
No reviewers directly named at this time - will name reviewers after I get to make some changes tomorrow. |
| researcher time and cost required to develop specific solutions, but it may not | ||
| lead to performance increases. | ||
|
|
||
| TODO: survival analysis/readmission prediction methods from EHR/EMR style data |
There was a problem hiding this comment.
Any preference for how long? How detailed? For now:
Nevertheless, recent work has revealed one domain in which deep networks have proven superior to traditional methods. Survival analysis models the time leading to an event of interest from a shared starting point, and in the context of EHR data, often associates these events to subject covariates. Exploring this relationship is difficult, however, given that EHR data types are heterogeneous, covariates are often missing, and conventional approaches require the covariate-event relationship be linear and aligned to a specific starting point (@arXiv:1608.02158). Early approaches, such as the Faraggi-Simon feed-forward network, aimed to relax the linearity assumption, but performance gains were lacking (@doi:10.1016/S0167-9473(99)00098-5). Katzman et al. in turn developed a deep implementation of the Faraggi-Simon network that, in addition to outperforming Cox regression, was capable of comparing the risk between a given pair of treatments, thus potentially acting as recommender system (@arXiv:1606.00931). To overcome the remaining difficulties, researchers have turned to deep exponential families, a class of latent generative models that are constructed from any type of exponential family distribution (@arXiv:1411.2581v1). This resulted in a deep survival analysis model capable of overcoming challenges posed by missing data and heterogeneous data types, while uncovering nonlinear relationships between covariates and failure time. They showed their model more accurately stratified patients as a function of disease risk score compared the current clinical implementation (@arXiv:1608.02158).
There was a problem hiding this comment.
This sounds good, I have nothing more to add, except for possibly mentioning other non-linear methods. I can think of Random Forest used in survival analysis (@arxivL0811.1645)
|
Ok - given that @sw1 has some content to contribute this is probably a good time to remove the [WIP] tag and aim to merge. This is a bit more of a laundry list than I like, but once we get the pieces in place we should return for some deeper analysis. This has some placeholders for expected contributions. |
cgreene
changed the title
|
Acknowledging the review request, I should be able to do it within 24 hours |
| @@ -137,3 +137,7 @@ interpretability problems, how can we best ensure reproducible models? What | |||
| might a clinician, or policy maker, need to see in a deep model in order to | |||
| influence healthcare decisions? Or, is deep learning a hypothesis generation | |||
| machine that requires manual validation?* | |||
|
|
|||
| ### Transfer learning/transferability of features | |||
There was a problem hiding this comment.
As you point out, transfer learning and multi-task learning has come up as a theme in many papers and domains. Do you suggest we defer most of that discussion in the domain-specific sections and coalesce it here? Or introduce it in the specific sections and have a cross-domain recap here?
There was a problem hiding this comment.
I guess I prefer to introduce in specific sections and cross-domain recap here. I think it's going to be key in bio for many near-future advances.
| @@ -19,7 +19,7 @@ tackle any of them? Are there example approaches whereby deep learning is | |||
| already having a transformational impact? I (Casey) have added some sections | |||
| below where I think we could contribute to the field with our discussion.* | |||
|
|
|||
| ### Major challenges | |||
| ### Major Areas of Existing Contributions | |||
There was a problem hiding this comment.
Should we settle on title case or lower case for section and sub-section headings?
There was a problem hiding this comment.
We should settle on one. I don't have a preference either way. What do you prefer? I suggest we make one quick PR to go through and match case as soon as we decide.
There was a problem hiding this comment.
Went with sentence case.
| all of this work, the researchers must work around a specific challenge - the | ||
| limited number of well annotated training images. To expand the number and | ||
| diversity of images, the researchers have employed approaches where they employ | ||
| adversarial examples [@doi:10.1101/095786] or first train towards human-created |
There was a problem hiding this comment.
If we haven't introduced adversarial networks earlier in the manuscript, it may deserve more attention.
There was a problem hiding this comment.
Definitely. These are adversarial examples though. They take the same base image and apply perturbations to it. I think you're probably thinking of something more like this - which is an adversarial network generated training example:
https://arxiv.org/pdf/1612.07828v1.pdf
There was a problem hiding this comment.
Got it. You're right, my mind jumped to Generative Adversarial Networks when I saw adversarial.
| limited number of well annotated training images. To expand the number and | ||
| diversity of images, the researchers have employed approaches where they employ | ||
| adversarial examples [@doi:10.1101/095786] or first train towards human-created | ||
| features before subsequent fine tuning [@doi:10.1007/978-3-319-46723-8_13]. The |
There was a problem hiding this comment.
What does "train towards human-created features" mean? It sounds interesting. Is this initializing weights using prior knowledge? Training a network with manually-defined features first and then using those weights to initialize a different network?
There was a problem hiding this comment.
There are some features that have been developed in the past for this task. They first perform supervised learning where the task is to regress output nodes (if I recall correctly) towards those features. Then, after that process, they fine tune specifically on their supervised examples. This is perhaps a more efficient way (in terms of # of examples) to build the network by pushing it towards intermediate features that are thought to be useful.
| with deep learning approaches. In recent work, Wang et al.[@arxiv:1606.05718] | ||
| analyzed stained slides to identify cancers within slides of lymph node slices. | ||
| The approach provided a probability map for each slide. On this task a | ||
| pathologist has about a 3% error rate. Their algorithm had about a 7% error |
There was a problem hiding this comment.
Do we need to be more precise or is this the terminology used in the paper? Does their error rate include false positives and false negatives?
There was a problem hiding this comment.
I think the language in the paper was pretty loose. The pathologist had no false positives, so theirs are all false negatives. For the algorithm it had the normal FP/FN tradeoff. I think we may want to be less precise here. I'll make that revision now.
| evaluated were unigrams and bigrams. These are the counts for single words and | ||
| two-word combinations in a free text document. They subset the full set of words | ||
| and word combinations to the 400 most commonly used ones. The machine learning | ||
| algorithms that they employed (naive bayes, logistic regression, and deep neural |
There was a problem hiding this comment.
"Naive Bayes" or "naive Bayes"?
|
|
||
| ##### Opportunities | ||
|
|
||
| However, significant work needs to be done to move these from conceptual |
There was a problem hiding this comment.
@cgreene are you planing to write these placeholder sections or are you looking for help here?
There was a problem hiding this comment.
At this point, bullets are probably good. Still have some more work to do on this but wanted to give @sw1 a chance to integrate his new text.
| Additionally, unique barriers exist in this space that may hinder progress in | ||
| this field. | ||
|
|
||
| ###### Data sharing and privacy? |
There was a problem hiding this comment.
@XieConnect are you planning to write something about this topic?
Pattern Discovery / Categorize
Motivation: We need to start filling in the sections with relevant literature.
Progress: This includes some thoughts on unsupervised EHR analysis and some extraction work via NLP. I have stubbed in portions for @sw1 and potentially @traversc. If you guys want to contribute you can either work of of this PR and make a pull request into this branch of my repo, or you can start writing now and create a PR against the repo once this goes through. My impression is that @sw1 is the lead on this portion given his previous volunteering.
Timing: I estimate that I'll get to wrap up the text around EHRs tomorrow. I just got a paper that I was struggling to get that seems relevant. I need some time to review it. I will also aim to talk some about the imaging work, but that may be a separate PR.