Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not enough p-values #68

Closed
tleonardi opened this issue Apr 5, 2019 · 2 comments
Closed

Not enough p-values #68

tleonardi opened this issue Apr 5, 2019 · 2 comments
Labels
bug

Comments

@tleonardi
Copy link
Owner

@tleonardi tleonardi commented Apr 5, 2019

Describe the bug
This error is raised also when using --min_ref_length 60. Related to #63.

nanocompore.common.NanocomporeError: Not enough p-values for a context
of order 2

Traceback

$ nanocompore -v
v1.0.0rc1

...

Starting data processing
  16%|#########2                                               | 371/2283
[24:12<2:40:01,  5.02s/ Processed References]Traceback (most recent call
last):
   File "/usr/local/bin/nanocompore", line 9, in <module>
     load_entry_point('nanocompore==1.0.0rc1', 'console_scripts',
'nanocompore')()
   File
"/usr/local/lib/python3.5/dist-packages/nanocompore/nanocompore_main.py",
line 142, in main
     args.func(args)
   File
"/usr/local/lib/python3.5/dist-packages/nanocompore/nanocompore_main.py",
line 177, in sampcomp_main
     db = s()
   File "/usr/local/lib/python3.5/dist-packages/nanocompore/SampComp.py",
line 253, in __call__
     raise E
   File "/usr/local/lib/python3.5/dist-packages/nanocompore/SampComp.py",
line 245, in __call__
     raise NanocomporeError(tb)
nanocompore.common.NanocomporeError: Traceback (most recent call last):
   File "/usr/local/lib/python3.5/dist-packages/nanocompore/SampComp.py",
line 368, in __process_references
     random_state=random_state)
   File "/usr/local/lib/python3.5/dist-packages/nanocompore/TxComp.py",
line 112, in txCompare
     corr_matrix_dict[test] = cross_corr_matrix(pval_list_dict[test],
sequence_context)
   File "/usr/local/lib/python3.5/dist-packages/nanocompore/TxComp.py",
line 319, in cross_corr_matrix
     raise NanocomporeError("Not enough p-values for a context of order
%s"%context)
nanocompore.common.NanocomporeError: Not enough p-values for a context
of order 2

@tleonardi tleonardi added the bug label Apr 5, 2019
@tleonardi

This comment has been minimized.

Copy link
Owner Author

@tleonardi tleonardi commented Apr 5, 2019

Ok, after looking into the issue it looks like the problem is caused by the lowCoverage check of TxComp.
Essentially, it's possible that for a transcript just above --min_coverage, only a few positions pass this threshold. If the number of passing positions is less than context*3+3 we get the Not enough p-values for a context of order 2 error.

Additionally, I've noticed that there's another error:

# Collect pvalue lists per tests
pval_list_dict = defaultdict(list)
for pos_dict in ref_pos_list:
    if 'txComp' in pos_dict:
        for test in tests:
           if test in pos_dict['txComp']:
               pval_list_dict[test].append(pos_dict['txComp'][test])

This block completely ignores the lowCov positions, so if we have something like:

0.01
lowCov
0.01
lowCov

pval_list_dict becomes: [0.01, 0.01], leading to a wrong cross correlation.

The solution is to set the p-values of the lowCov positions to 1 when filling the pval_list_dict list.

@a-slide

This comment has been minimized.

Copy link
Collaborator

@a-slide a-slide commented Apr 5, 2019

Sounds like a reasonable approach for the lowCov.

tleonardi added a commit that referenced this issue Apr 9, 2019
tleonardi added a commit that referenced this issue Apr 9, 2019
a-slide added a commit that referenced this issue Apr 9, 2019
Fixed issue #68
@tleonardi tleonardi closed this Apr 10, 2019
tleonardi added a commit that referenced this issue Apr 10, 2019
* Added empty changelog

* Small doc changes

* Typo and reformat doc

* Fixed issue #68

* Added travis file

* Added branches safelist

* Removed python 3.2

* Added devel branch to travis safelist

* Updated pytest version

* Replcated tmpdir_factory with tmp_path_factory

* Fixed python version compatibility

* Fixed syntax error

* Casting pathlib objects to string for compatibility with python3.5

* Fixed python 3.5 compatibility error

* Fixed issue #68

* Removed trailing whitespace

* Added travis badge

* Skip many integration tests if running inside Travis

* Added conditional workflow for 3.5 version

* Lowered p-value tolerance for python 3.5

* Fixed typos in checking python version

* nbconvert is now installed bebore mknotebooks

* Further tolerance adjustment for python3.5

* Updated Changelog

* Clean up of travis yml file

* Added Slack notifications and automatic gh-pages deployment (#78)

* Updated release data and bumped version number

* Add Changelog to doc (#81)

* Workaround for issue #82
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.