strict & fixing Docs (#24)

* fix sphinx docs
* sphinx versions
* fix sphinx warn
* optimize imports
* add doc ignores
* fix docs paths
* fix notebook
* docs: drop SVG & GIF

MANIFEST.in

@@ -34,7 +34,7 @@ recursive-include experiments_* *.py *.yml
 recursive-include handling_annotations *.py
 
 # Include the figures
-recursive-include figures *.gif *.jpg *.png
+recursive-include docs/source/_figures *.gif *.jpg *.png
 
 # Include the sample images
 include data_images/others/lena.png

README.md

@@ -17,9 +17,11 @@
 
 ## Superpixel segmentation with GraphCut regularisation
 
-Image segmentation is widely used as an initial phase of many image processing tasks in computer vision and image analysis. Many recent segmentation methods use superpixels because they reduce the size of the segmentation problem by order of magnitude. Also,  features on superpixels are much more robust than features on pixels only. We use spatial regularisation on superpixels to make segmented regions more compact. The segmentation pipeline comprises (i) computation of superpixels; (ii) extraction of descriptors such as colour and texture; (iii) soft classification, using a standard classifier for supervised learning, or the Gaussian Mixture Model for unsupervised learning; (iv) final segmentation using Graph Cut. We use this segmentation pipeline on real-world applications in medical imaging (see a sample [images](./data_images)). We also show that [unsupervised segmentation](notebooks/segment-2d_slic-fts-clust-gc.ipynb) is sufficient for some situations, and provides similar results to those obtained using [trained segmentation](notebooks/segment-2d_slic-fts-classif-gc.ipynb).
+Image segmentation is widely used as an initial phase of many image processing tasks in computer vision and image analysis. Many recent segmentation methods use superpixels because they reduce the size of the segmentation problem by order of magnitude. Also,  features on superpixels are much more robust than features on pixels only. We use spatial regularisation on superpixels to make segmented regions more compact. The segmentation pipeline comprises (i) computation of superpixels; (ii) extraction of descriptors such as colour and texture; (iii) soft classification, using a standard classifier for supervised learning, or the Gaussian Mixture Model for unsupervised learning; (iv) final segmentation using Graph Cut. We use this segmentation pipeline on real-world applications in medical imaging (see [sample images](data_images/).
+ We also show that [unsupervised segmentation](notebooks/segment-2d_slic-fts-clust-gc.ipynb) is sufficient for some situations,
+  and provides similar results to those obtained using [trained segmentation](notebooks/segment-2d_slic-fts-classif-gc.ipynb).
 
-![schema](figures/schema_slic-fts-clf-gc.jpg)
+![schema](docs/source/_figures/schema_slic-fts-clf-gc.jpg)
 
 **Sample ipython notebooks:**
 * [Supervised segmentation](notebooks/segment-2d_slic-fts-classif-gc.ipynb) requires training annotation
@@ -28,24 +30,25 @@ Image segmentation is widely used as an initial phase of many image processing t
 
 **Illustration**
 
-![input image](figures/insitu7545.jpg)
-![segmentation](figures/insitu7545_gc.png)
+![input image](docs/source/_figures/insitu7545.jpg)
+![segmentation](docs/source/_figures/insitu7545_gc.png)
 
 Reference: _Borovec J., Svihlik J., Kybic J., Habart D. (2017). **Supervised and unsupervised segmentation using superpixels, model estimation, and Graph Cut.** In: Journal of Electronic Imaging._
 
 
 ## Object centre detection and Ellipse approximation
 
-An image processing pipeline to detect and localize Drosophila egg chambers that consists of the following steps: (i) superpixel-based image segmentation into relevant tissue classes (see above); (ii) detection of egg center candidates using label histograms and ray features; (iii) clustering of center candidates and; (iv) area-based maximum likelihood ellipse model fitting. See our [Poster](http://cmp.felk.cvut.cz/~borovji3/documents/poster-MLMI2017.compressed.pdf) related to this work.
+An image processing pipeline to detect and localize Drosophila egg chambers that consists of the following steps: (i) superpixel-based image segmentation into relevant tissue classes (see above); (ii) detection of egg center candidates using label histograms and ray features; (iii) clustering of center candidates and; (iv) area-based maximum likelihood ellipse model fitting.
+ See our [Poster](http://cmp.felk.cvut.cz/~borovji3/documents/poster-MLMI2017.compressed.pdf) related to this work.
 
 **Sample ipython notebooks:**
 * [Center detection](notebooks/egg-center_candidates-clustering.ipynb) consists of center candidate training and prediction, and candidate clustering.
 * [Ellipse fitting](notebooks/egg-detect_ellipse-fitting.ipynb) with given estimated center structure segmentation.
 
 **Illustration**
 
-![estimated centres](figures/insitu7545_center_clusters.jpg)
-![ellipse fitting](figures/insitu7545_ellipse_fit.png)
+![estimated centres](docs/source/_figures/insitu7545_center_clusters.jpg)
+![ellipse fitting](docs/source/_figures/insitu7545_ellipse_fit.png)
 
 Reference: _Borovec J., Kybic J., Nava R. (2017) **Detection and Localization of Drosophila Egg Chambers in Microscopy Images.** In: Machine Learning in Medical Imaging._
 
@@ -60,8 +63,8 @@ Region growing is a classical image segmentation method based on hierarchical re
 
 **Illustration**
 
-![growing RG](figures/insitu7545_RG2SP_animation.gif)
-![ellipse fitting](figures/insitu7545_RG2SP-gc-mm.png)
+![growing RG](docs/source/_figures/insitu7545_RG2SP_animation.gif)
+![ellipse fitting](docs/source/_figures/insitu7545_RG2SP-gc-mm.png)
 
 Reference: _Borovec J., Kybic J., Sugimoto, A. (2017). **Region growing using superpixels with learned shape prior.** In: Journal of Electronic Imaging._
 
@@ -164,7 +167,7 @@ We introduce some useful tools for work with image annotation and segmentation.
 ### Semantic (un/semi)supervised segmentation
 
 We utilise (un)supervised segmentation according to given training examples or some expectations.
-![vusial debug](figures/visual_img_43_debug.jpg)
+![vusial debug](docs/source/_figures/visual_img_43_debug.jpg)
 
 * Evaluate superpixels (with given SLIC parameters) quality against given segmentation. It helps to find out the best SLIC configuration.
     ```bash
@@ -188,7 +191,7 @@ We utilise (un)supervised segmentation according to given training examples or s
        -cfg ./experiments_segmentation/sample_config.yml \
        -o ./results -n langIsl --nb_classes 3 --visual --nb_workers 2
     ```
-    ![unsupervised](figures/imag-disk-20_gmm.jpg)
+    ![unsupervised](docs/source/_figures/imag-disk-20_gmm.jpg)
 * Perform **Supervised** segmentation with afterwards evaluation.
     ```bash
     python experiments_segmentation/run_segm_slic_classif_graphcut.py \
@@ -197,7 +200,7 @@ We utilise (un)supervised segmentation according to given training examples or s
         --path_config ./experiments_segmentation/sample_config.yml \
         -o ./results -n Ovary --img_type 2d_split --visual --nb_workers 2
     ```
-    ![supervised](figures/imag-disk-20_train.jpg)
+    ![supervised](docs/source/_figures/imag-disk-20_train.jpg)
 * Perform **Semi-Supervised** is using the the supervised pipeline with not fully annotated images.
 * For both experiment you can evaluate segmentation results.
     ```bash
@@ -207,7 +210,7 @@ We utilise (un)supervised segmentation according to given training examples or s
         -i "./data_images/drosophila_ovary_slice/image/*.jpg" \
         -o ./results/evaluation --visual
     ```
-    ![vusial](figures/segm-visual_D03_sy04_100x.jpg)
+    ![vusial](docs/source/_figures/segm-visual_D03_sy04_100x.jpg)
 
 The previous two (un)segmentation accept [configuration file](experiments_segmentation/sample_config.yml) (YAML) by parameter `-cfg` with some extra parameters which was not passed in arguments, for instance:
 ```yaml
@@ -283,7 +286,7 @@ In general, the input is a formatted list (CSV file) of input images and annotat
         -out ~/Medical-drosophila/RESULTS/atlas_datasets/ovary_images/stage_3
     ```
 
-![ellipse fitting](figures/insitu7544_ellipses.jpg)
+![ellipse fitting](docs/source/_figures/insitu7544_ellipses.jpg)
 
 ### Region growing with a shape prior
 
@@ -333,7 +336,7 @@ pip install --user git+https://github.com/Borda/morph-snakes.git
     ```bash
     python experiments_ovary_detect/run_export_user-annot-segm.py
     ```
-    ![user-annnot](figures/insitu7545_user-annot-segm.jpg)
+    ![user-annnot](docs/source/_figures/insitu7545_user-annot-segm.jpg)
 
 ---
 

circle.yml

@@ -45,8 +45,12 @@ references:
      command: |
        sudo apt-get install pandoc
        sudo pip install -r docs/requirements.txt
-       # sphinx-apidoc -o ./docs/source ./imsegm ./tests/* --force --follow-links
-       cd docs; make html
+       # sphinx-apidoc -o ./docs/source ./imsegm **/test_* --force --follow-links
+       python setup.py build_ext --inplace
+       cd docs
+       make html --debug --jobs 2 SPHINXOPTS="-W"
+       make latexpdf
+
 
 jobs:
   Py3-Tests:
@@ -61,8 +65,6 @@ jobs:
       - *build_project
       # TESTING
       - *test_coverage
-      # DOCUMENTATION
-      - *make_docs
 
       # PASSING
       - run:
@@ -106,6 +108,21 @@ jobs:
       - image: circleci/python:2.7
     steps: *steps_expt
 
+  Build-Docs:
+    docker:
+      - image: circleci/python:3.6
+    steps:
+      - checkout
+      - run:
+          name: TexLive
+          command: |
+            sudo apt-get update -qq
+            sudo apt-get install -y imagemagick ghostscript latexmk texlive texlive-latex-recommended texlive-fonts-recommended texlive-formats-extra
+      # INSTALLATION
+      - *install_pips
+      # DOCUMENTATION
+      - *make_docs
+
 workflows:
   version: 2
   build:
@@ -114,3 +131,4 @@ workflows:
       - Py3-Tests
       - Py2-Experiments
       - Py3-Experiments
+      - Build-Docs

docs/requirements.txt

@@ -1,6 +1,7 @@
-sphinx>=1.4
+sphinx>=2.0, <3.0
 recommonmark  # fails with badges
 m2r  # fails with multi-line text
 nbsphinx
 pandoc
-docutils<0.15  # higher version breaks py2
+docutils<0.15  # higher version breaks py2
+ipython

docs/source/conf.py

@@ -12,11 +12,12 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 
-import os
-import sys
 import glob
-import shutil
 import inspect
+import os
+import shutil
+import sys
+import re
 
 import m2r
 
@@ -26,6 +27,22 @@
 
 import imsegm  # noqa: E402
 
+# -- Project information -----------------------------------------------------
+
+project = 'ImSegm'
+copyright = imsegm.__copyright__
+author = imsegm.__author__
+
+# The short X.Y version
+version = imsegm.__version__
+# The full version, including alpha/beta/rc tags
+release = imsegm.__version__
+
+# Options for the linkcode extension
+# ----------------------------------
+github_user = 'Borda'
+github_repo = 'pyImSegm'
+
 # -- Project documents -------------------------------------------------------
 
 # export the documentation
@@ -38,29 +55,25 @@
 with open(os.path.join(PATH_ROOT, 'README.md'), 'r') as fp:
     readme = fp.read()
 # replace all paths to relative
-for ndir in (os.path.basename(p) for p in glob.glob(os.path.join(PATH_ROOT, '*'))
-             if os.path.isdir(p)):
-    readme = readme.replace('](%s/' % ndir, '](%s/%s/' % (PATH_ROOT, ndir))
+readme = readme.replace('](docs/source/', '](')
+readme = re.sub(r' \[(.*)\]\((?!http)(.*)\)',
+                r' [\1](https://github.com/%s/%s/blob/master/\2)' % (github_user, github_repo),
+                readme)
+# TODO: temp fix removing SVG badges and GIF, because PDF cannot show them
+readme = re.sub(r'(\[!\[.*\))', '', readme)
+readme = re.sub(r'(!\[.*.gif\))', '', readme)
+# for dir_name in (os.path.basename(p) for p in glob.glob(os.path.join(PATH_ROOT, '*'))
+#                  if os.path.isdir(p)):
+#     readme = readme.replace('](%s/' % dir_name, '](%s/%s/' % (PATH_ROOT, dir_name))
 with open('readme.md', 'w') as fp:
     fp.write(readme)
 
-# -- Project information -----------------------------------------------------
-
-project = 'ImSegm'
-copyright = imsegm.__copyright__
-author = imsegm.__author__
-
-# The short X.Y version
-version = imsegm.__version__
-# The full version, including alpha/beta/rc tags
-release = imsegm.__version__
-
 
 # -- General configuration ---------------------------------------------------
 
 # If your documentation needs a minimal Sphinx version, state it here.
 
-needs_sphinx = '1.4'
+needs_sphinx = '2.2'
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
@@ -75,6 +88,7 @@
     'sphinx.ext.linkcode',
     'sphinx.ext.napoleon',
     'sphinx.ext.autosummary',
+    # 'sphinxcontrib.rsvgconverter'
     'recommonmark',
     # 'm2r',
     'nbsphinx',
@@ -109,7 +123,13 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['*tests.*', '*.test_*']
+exclude_patterns = [
+    'data_images',
+    '*tests.*', '*.test_*',
+    '*.so', '*.dll',
+    'modules.rst',
+    '*/transform-img-plane_inter-circle.ipynb'
+]
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = None
@@ -131,7 +151,7 @@
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
+html_static_path = ['_figures']  # , '_static', 'notebooks'
 
 # Custom sidebar templates, must be a dictionary that maps document names
 # to template names.
@@ -255,12 +275,12 @@ def setup(app):
 
 
 # copy all notebooks to local folder
-path_nbs = os.path.join(PATH_HERE, 'notebooks')
-if not os.path.isdir(path_nbs):
-    os.mkdir(path_nbs)
-for path_ipynb in glob.glob(os.path.join(PATH_ROOT, 'notebooks', '*.ipynb')):
-    path_ipynb2 = os.path.join(path_nbs, os.path.basename(path_ipynb))
-    shutil.copy(path_ipynb, path_ipynb2)
+path_docs_nbs = os.path.join(PATH_HERE, 'notebooks')
+if not os.path.isdir(path_docs_nbs):
+    os.mkdir(path_docs_nbs)
+for path_ipynb_in in glob.glob(os.path.join(PATH_ROOT, 'notebooks', '*.ipynb')):
+    path_ipynb_new = os.path.join(path_docs_nbs, os.path.basename(path_ipynb_in))
+    shutil.copy(path_ipynb_in, path_ipynb_new)
 
 
 # Ignoring Third-party packages
@@ -278,12 +298,6 @@ def setup(app):
 autodoc_mock_imports = MOCK_MODULES + ['sklearn', 'skimage', 'gco', 'yaml']
 
 
-# Options for the linkcode extension
-# ----------------------------------
-github_user = 'Borda'
-github_repo = 'pyImSegm'
-
-
 # Resolve function
 # This function is used to populate the (source) links in the API
 def linkcode_resolve(domain, info):

docs/source/examples.rst

@@ -13,4 +13,4 @@ Examples
 
    notebooks/egg_segment_graphcut.ipynb
    notebooks/egg-center_candidates-clustering.ipynb
-   notebooks/egg-detect_ellipse-fitting.ipynb
+   notebooks/egg-detect_ellipse-fitting.ipynb

experiments_ovary_centres/gui_annot_center_correction.py

@@ -26,18 +26,19 @@
 Copyright (C) 2016 Jiri Borovec <jiri.borovec@fel.cvut.cz>
 """
 
-import os
-import sys
+import argparse
 import glob
 import logging
-import argparse
+import os
+import sys
 
+# from planar import line as pl_line
+import matplotlib
+import matplotlib.pylab as plt
 import numpy as np
 import pandas as pd
-import matplotlib.pylab as plt
 from scipy import ndimage, spatial
-# from planar import line as pl_line
-import matplotlib
+
 matplotlib.use('GTKAgg')  # or >> matplotlib.rcsetup.all_backends
 
 

experiments_ovary_centres/run_center_candidate_training.py

@@ -24,18 +24,18 @@
 Copyright (C) 2016-2017 Jiri Borovec <jiri.borovec@fel.cvut.cz>
 """
 
+import argparse
+import logging
 import os
 import sys
-import logging
-import argparse
 from functools import partial
 
-import tqdm
-import pandas as pd
+import matplotlib
 import numpy as np
+import pandas as pd
+import tqdm
 from scipy import spatial
 
-import matplotlib
 if os.environ.get('DISPLAY', '') == '' and matplotlib.rcParams['backend'] != 'agg':
     print('No display found. Using non-interactive Agg backend.')
     matplotlib.use('Agg')