DOC transforming minimal notebook to become example by maikia · Pull Request #33 · scverse/PyDESeq2

maikia · 2023-01-05T14:45:51Z

introducing sphinx-gallery with the first example converted from the notebooks folder.

Note:

some more explanations might be added in the future for more novice users
if accepted the second example will be added as a separate PR
since this is redundant with the notebooks sections we should consider how in the future make them distinct (eg notebooks used for binder and examples with a good through explanation)

Here how it looks like now (locally):

arthurPignetOwkin · 2023-01-05T16:47:37Z

Hello Maria. This PR is not working on RTD, the compilation is failing.

Running Sphinx v4.4.0
loading translations [en]... done
making output directory... done
myst v0.16.1: MdParserConfig(renderer='sphinx', commonmark_only=False, enable_extensions=['dollarmath'], linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], url_schemes=['http', 'https', 'mailto', 'ftp'], heading_anchors=None, heading_slug_func=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'], words_per_minute=200)
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://numpy.org/doc/stable/objects.inv...
loading intersphinx inventory from https://pandas.pydata.org/docs/objects.inv...
generating gallery...
Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files.
generating gallery for auto_examples... [100%] plot_minimal_pydeseq2_pipeline.py
WARNING: /home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py failed to execute correctly: Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py", line 47, in <module>
    counts_df = load_data(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pydeseq2/utils.py", line 79, in load_data
    df = pd.read_csv(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1735, in _make_engine
    self.handles = get_handle(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/common.py", line 856, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/tests/data/test_counts.csv'


computation time summary:
    - ../../examples/plot_minimal_pydeseq2_pipeline.py:   0.83 sec   0.0 MB
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 11 source files that are out of date
updating environment: [new config] 14 added, 0 changed, 0 removed
reading sources... [  7%] api/DeseqDataSet
reading sources... [ 14%] api/DeseqStats
reading sources... [ 21%] api/grid_search
reading sources... [ 28%] api/index
reading sources... [ 35%] api/preprocessing
reading sources... [ 42%] api/utils
reading sources... [ 50%] auto_examples/index
reading sources... [ 57%] auto_examples/plot_minimal_pydeseq2_pipeline
reading sources... [ 64%] auto_examples/sg_execution_times
reading sources... [ 71%] index
reading sources... [ 78%] usage/contributing
reading sources... [ 85%] usage/getting_started
reading sources... [ 92%] usage/installation
reading sources... [100%] usage/requirements

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  7%] api/DeseqDataSet
writing output... [ 14%] api/DeseqStats
writing output... [ 21%] api/grid_search
writing output... [ 28%] api/index
writing output... [ 35%] api/preprocessing
writing output... [ 42%] api/utils
writing output... [ 50%] auto_examples/index
writing output... [ 57%] auto_examples/plot_minimal_pydeseq2_pipeline
writing output... [ 64%] auto_examples/sg_execution_times
writing output... [ 71%] index
writing output... [ 78%] usage/contributing
writing output... [ 85%] usage/getting_started
writing output... [ 92%] usage/installation
writing output... [100%] usage/requirements

generating indices... genindex py-modindex done
writing additional pages... search done
copying images... [100%] auto_examples/images/thumb/sphx_glr_plot_minimal_pydeseq2_pipeline_thumb.png

copying downloadable files... [ 25%] auto_examples/auto_examples_python.zip
copying downloadable files... [ 50%] auto_examples/auto_examples_jupyter.zip
copying downloadable files... [ 75%] auto_examples/plot_minimal_pydeseq2_pipeline.py
copying downloadable files... [100%] auto_examples/plot_minimal_pydeseq2_pipeline.ipynb

copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in _build/html.
Updating searchtools for Read the Docs search... 
Sphinx-Gallery successfully executed 0 out of 1 file subselected by:

    gallery_conf["filename_pattern"] = '/plot'
    gallery_conf["ignore_pattern"]   = '__init__\\.py'

after excluding 0 files that had previously been run (based on MD5).


Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/sphinx/cmd/build.py", line 284, in build_main
    app.build(args.force_all, filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/sphinx/application.py", line 374, in build
    self.events.emit('build-finished', None)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/sphinx/events.py", line 101, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/sphinx_gallery/gen_gallery.py", line 852, in summarize_failing_examples
    raise ExtensionError(fail_message)
sphinx.errors.ExtensionError: Here is a summary of the problems encountered when running the examples

Unexpected failing examples:
/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py", line 47, in <module>
    counts_df = load_data(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pydeseq2/utils.py", line 79, in load_data
    df = pd.read_csv(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1735, in _make_engine
    self.handles = get_handle(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/common.py", line 856, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/tests/data/test_counts.csv'


-------------------------------------------------------------------------------

Extension error:
Here is a summary of the problems encountered when running the examples

Unexpected failing examples:
/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/checkouts/examples/examples/plot_minimal_pydeseq2_pipeline.py", line 47, in <module>
    counts_df = load_data(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pydeseq2/utils.py", line 79, in load_data
    df = pd.read_csv(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/util/_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1735, in _make_engine
    self.handles = get_handle(
  File "/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/pandas/io/common.py", line 856, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/home/docs/checkouts/readthedocs.org/user_builds/pydeseq2/envs/examples/lib/python3.9/site-packages/tests/data/test_counts.csv'


-------------------------------------------------------------------------------
Durée de la commande : 8s Code de retour: 2

grst · 2023-01-05T17:04:37Z

it might be useful to enable PR builds on readthedocs: https://docs.readthedocs.io/en/stable/pull-requests.html

That way you get feedback on each PR if it can be successfully built and also get a preview of the new documentation if it was successful.

maikia · 2023-01-06T12:41:39Z

Thanks @arthurPignetOwkin , indeed RTD didn't see the data path. I added a selection that if the path is not found the data file is loaded directly from github. It should work now

BorisMuzellec · 2023-01-09T13:15:38Z

Hi, thanks a lot for your work @maikia. For some reason my RTD account can't be added to the pydeseq2 project. I've contacted the RTD support on this issue, there should be a fixed released this week. Once this is fixed I should be able to review your PR.

jeandut · 2023-01-10T10:53:38Z

"PYDESEQ2 PIPELINE" title is too big it's threatening.

maikia · 2023-01-10T11:03:15Z

"PYDESEQ2 PIPELINE" title is too big it's threatening.

ops, sorry. updated comment: I though you were referring to the title of the documentation and not the notebook. I will include this review comment in the revision. Thanks for the suggestion

BorisMuzellec · 2023-01-13T09:57:55Z

Hi @maikia, I rebased main on this branch to resolve merge conflicts. Will review this PR ASAP.

BorisMuzellec

Thanks for the nice work @maikia, the example looks great!

I've just suggested a few changes to the contents of the example itself: I don't think it's relevant to mention TCGA data (since we're not giving precise instructions to download it) nor to keep the SAVE flag and output pickling. Let me know what you think about this.

examples/plot_minimal_pydeseq2_pipeline.py

BorisMuzellec · 2023-01-13T10:12:18Z

examples/plot_minimal_pydeseq2_pipeline.py

+dds = DeseqDataSet(
+    counts_df,
+    clinical_df,
+    design_factors="condition" if DATASET == "synthetic" else "high_grade",


Suggested change

design_factors="condition" if DATASET == "synthetic" else "high_grade",

design_factors="condition",

again, I think this should be explained, not removed?

Not sure, the high_grade field is specific to TCGA data. We could add a comment saying that "condition" is a column of clinical_df

examples/plot_minimal_pydeseq2_pipeline.py

maikia · 2023-01-13T14:47:02Z

@BorisMuzellec Perhaps we should re-discuss the purpose of this example? In view of your comments it seems that we had a different idea :-)

BorisMuzellec

@maikia I've updated my review following our conversation: let's keep the gene-filtering and output-saving parts, but remove TCGA.

examples/plot_minimal_pydeseq2_pipeline.py

maikia · 2023-01-16T16:35:29Z

Thanks @BorisMuzellec for your review.
I think this example needs another update (to add more comments and explanations for newbies). However, I would prefer to keep it as a separate PR. This PR is to introduce examples in the docs and convert first existing example as is. What do you think?

maikia · 2023-01-16T16:37:30Z

A new look:

…tly from github (rebase and fix conflict with load_example_data refactoring)

Co-authored-by: Boris Muzellec <BorisMuzellec@users.noreply.github.com>

BorisMuzellec

Hi @maikia, thanks a lot for the great work!
I agree that there is still some work to do to improve the example (we should add further explanations) but I think we should merge this PR, which already implements the notebook rendering in the docs, and save further improvements on the text for a later PR.

maikia requested review from BorisMuzellec and arthurPignetOwkin January 5, 2023 14:45

maikia changed the title ~~transforming minimal notebook to become example~~ DOC transforming minimal notebook to become example Jan 5, 2023

maikia force-pushed the examples branch from eca6b4b to f89394e Compare January 11, 2023 09:38

BorisMuzellec added the documentation Improvements or additions to documentation label Jan 12, 2023

BorisMuzellec force-pushed the examples branch from b11408d to 044a5a2 Compare January 13, 2023 09:31

BorisMuzellec requested changes Jan 13, 2023

View reviewed changes

BorisMuzellec requested changes Jan 16, 2023

View reviewed changes

examples/plot_minimal_pydeseq2_pipeline.py Show resolved Hide resolved

maikia force-pushed the examples branch from 2e7eb4c to c06d0cf Compare January 16, 2023 16:17

maikia requested a review from BorisMuzellec January 16, 2023 16:35

maikia and others added 10 commits January 17, 2023 09:52

transforming minimal notebook to become example

38d988a

add the data to source

b0a41a3

if the data directory for synthetic data does not exist loading direc…

147bce5

…tly from github (rebase and fix conflict with load_example_data refactoring)

update function name for data loading

fba1744

update design_factors

7b4f324

refactor: change load_example_data "cancer_type" argument to "dataset"

396d452

Update examples/plot_minimal_pydeseq2_pipeline.py

36841e9

Co-authored-by: Boris Muzellec <BorisMuzellec@users.noreply.github.com>

Update examples/plot_minimal_pydeseq2_pipeline.py

1b6283b

Co-authored-by: Boris Muzellec <BorisMuzellec@users.noreply.github.com>

Update examples/plot_minimal_pydeseq2_pipeline.py

ef63c21

Co-authored-by: Boris Muzellec <BorisMuzellec@users.noreply.github.com>

Update examples/plot_minimal_pydeseq2_pipeline.py

e52f38b

Co-authored-by: Boris Muzellec <BorisMuzellec@users.noreply.github.com>

maikia added 3 commits January 17, 2023 09:52

rephrasing

9562dad

add some comments

7941223

merge

2d6d215

maikia force-pushed the examples branch from ad23502 to 2d6d215 Compare January 17, 2023 08:52

BorisMuzellec approved these changes Jan 17, 2023

View reviewed changes

maikia merged commit a781d62 into main Jan 17, 2023

maikia deleted the examples branch January 17, 2023 09:00

	design_factors="condition" if DATASET == "synthetic" else "high_grade",
	design_factors="condition",

Conversation

maikia commented Jan 5, 2023

Uh oh!

arthurPignetOwkin commented Jan 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grst commented Jan 5, 2023

Uh oh!

maikia commented Jan 6, 2023

Uh oh!

BorisMuzellec commented Jan 9, 2023

Uh oh!

jeandut commented Jan 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maikia commented Jan 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BorisMuzellec commented Jan 13, 2023

Uh oh!

BorisMuzellec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BorisMuzellec Jan 13, 2023

Choose a reason for hiding this comment

Uh oh!

maikia Jan 13, 2023

Choose a reason for hiding this comment

Uh oh!

BorisMuzellec Jan 16, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maikia commented Jan 13, 2023

Uh oh!

BorisMuzellec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maikia commented Jan 16, 2023

Uh oh!

maikia commented Jan 16, 2023

Uh oh!

BorisMuzellec left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

arthurPignetOwkin commented Jan 5, 2023 •

edited

Loading

jeandut commented Jan 10, 2023 •

edited

Loading

maikia commented Jan 10, 2023 •

edited

Loading