Separation based on p-value or abs(mean) unrealistically good #5
Comments
tlnagy
added a commit
that referenced
this issue
Apr 30, 2016
Working towards a solution to #5. Dropping the coverage of library introduced a couple artifacts due to guides being absent starting at the transfection state.
|
Dropping coverage from 1000x to 100x and increasing the variance due to sorting to 1 gives the following plots... Observed Phenotypes
Volcano plot
It's interesting to note that there seem to be binning occuring when pvalue -> 0 Roc
|
|
@martinkampmann Plotting the ROC curves separately for linear and sigmoidal genes: This is with 100x and sigma=1
|
tlnagy
added a commit
that referenced
this issue
May 2, 2016
|
AUROC's for optimum screen conditions are good, but I can effectively degrade them so I'm going to go ahead and close this issue for now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Even with the new method introduced in f25e71e of sampling an "observed phenotype" from a Normal distribution centered at the "theoretical phenotype" with a stddev of 0.5, the separation is unrealistically good. Here I'm sorting guides based on their "observed phenotype" and taking the bottom 1/3 and putting it in one bin and the top 1/3 in another and comparing:
Distribution of "observed phenotypes" that was used to bin
Log-log plot of guide frequencies in the two bins
Volcano plot
It's super apparent when I collapse the results down to the gene-level:
What do you think @martinkampmann? I'm ending up with AUROC's of 0.999.
The text was updated successfully, but these errors were encountered: