Merge branch 'release/1.2-dev' into 5000

Lightning-AI · Jan 26, 2021 · b60982a · b60982a
2 parents eff2e81 + dee5553
commit b60982a
Show file tree

Hide file tree

Showing 56 changed files with 180 additions and 165 deletions.
diff --git a/.github/workflows/docs-checks.yml b/.github/workflows/docs-checks.yml
@@ -100,7 +100,7 @@ jobs:
           # First run the same pipeline as Read-The-Docs
           cd docs
           make clean
-          make html --debug --jobs 2 SPHINXOPTS="-W"
+          make html --debug --jobs $(nproc) SPHINXOPTS="-W"
 
       - name: Upload built docs
         uses: actions/upload-artifact@v2

diff --git a/.gitignore b/.gitignore
@@ -16,6 +16,7 @@ test_tube_exp/
 docs/source/api
 docs/source/*.md
 docs/source/generated
+docs/source/*/generated
 
 # Byte-compiled / optimized / DLL files
 __pycache__/

diff --git a/docs/source/amp.rst → docs/source/advanced/amp.rst b/docs/source/amp.rst → docs/source/advanced/amp.rst
diff --git a/docs/source/lr_finder.rst → docs/source/advanced/lr_finder.rst b/docs/source/lr_finder.rst → docs/source/advanced/lr_finder.rst
@@ -37,10 +37,10 @@ initial lr.
 Using Lightning's built-in LR finder
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To enable the learning rate finder, your :ref:`lightning_module` needs to have a ``learning_rate`` or ``lr`` property.
+To enable the learning rate finder, your :doc:`lightning module <../common/lightning_module>` needs to have a ``learning_rate`` or ``lr`` property.
 Then, set ``Trainer(auto_lr_find=True)`` during trainer construction,
 and then call ``trainer.tune(model)`` to run the LR finder. The suggested ``learning_rate``
-will be written to the console and will be automatically set to your :ref:`lightning_module`,
+will be written to the console and will be automatically set to your :doc:`lightning module <../common/lightning_module>`,
 which can be accessed via ``self.learning_rate`` or ``self.lr``.
 
 .. code-block:: python
@@ -106,7 +106,7 @@ below. It is recommended to not pick the learning rate that achieves the lowest
 loss, but instead something in the middle of the sharpest downward slope (red point).
 This is the point returned py ``lr_finder.suggestion()``.
 
-.. figure:: /_images/trainer/lr_finder.png
+.. figure:: ../_images/trainer/lr_finder.png
 
 The parameters of the algorithm can be seen below.
 

diff --git a/docs/source/multi_gpu.rst → docs/source/advanced/multi_gpu.rst b/docs/source/multi_gpu.rst → docs/source/advanced/multi_gpu.rst
@@ -115,7 +115,7 @@ When running in distributed mode, we have to ensure that the validation and test
 This is done by adding ``sync_dist=True`` to all ``self.log`` calls in the validation and test step.
 This ensures that each GPU worker has the same behaviour when tracking model checkpoints, which is important for later downstream tasks such as testing the best checkpoint across all workers.
 
-Note if you use any built in metrics or custom metrics that use the :ref:`Metrics API <metrics>`, these do not need to be updated and are automatically handled for you.
+Note if you use any built in metrics or custom metrics that use the :doc:`Metrics API <../extensions/metrics>`, these do not need to be updated and are automatically handled for you.
 
 .. testcode::
 
@@ -237,7 +237,7 @@ Note in particular the difference between `gpus=0`, `gpus=[0]` and `gpus="0"`.
     ``auto_select_gpus=True`` will automatically help you find ``k`` gpus that are not
     occupied by other processes. This is especially useful when GPUs are configured
     to be in "exclusive mode", such that only one process at a time can access them.
-    For more details see the :ref:`Trainer guide <trainer>`.
+    For more details see the :doc:`trainer guide <../common/trainer>`.
 
 ----------
 

diff --git a/docs/source/multiple_loaders.rst → docs/source/advanced/multiple_loaders.rst b/docs/source/multiple_loaders.rst → docs/source/advanced/multiple_loaders.rst
diff --git a/docs/source/profiler.rst → docs/source/advanced/profiler.rst b/docs/source/profiler.rst → docs/source/advanced/profiler.rst
diff --git a/docs/source/sequences.rst → docs/source/advanced/sequences.rst b/docs/source/sequences.rst → docs/source/advanced/sequences.rst
diff --git a/docs/source/tpu.rst → docs/source/advanced/tpu.rst b/docs/source/tpu.rst → docs/source/advanced/tpu.rst
diff --git a/docs/source/training_tricks.rst → docs/source/advanced/training_tricks.rst b/docs/source/training_tricks.rst → docs/source/advanced/training_tricks.rst
diff --git a/docs/source/transfer_learning.rst → docs/source/advanced/transfer_learning.rst b/docs/source/transfer_learning.rst → docs/source/advanced/transfer_learning.rst
diff --git a/docs/source/benchmarking.rst → docs/source/benchmarking/benchmarks.rst b/docs/source/benchmarking.rst → docs/source/benchmarking/benchmarks.rst
@@ -9,6 +9,6 @@ Time comparison
 We have set regular benchmarking against PyTorch vanilla training loop on with RNN and simple MNIST classifier as per of out CI.
 In average for simple MNIST CNN classifier we are only about 0.06s slower per epoch, see detail chart bellow.
 
-.. figure:: _images/benchmarks/figure-parity-times.png
+.. figure:: ../_images/benchmarks/figure-parity-times.png
    :alt: Speed parity to vanilla PT, created on 2020-12-16
    :width: 500
diff --git a/docs/source/performance.rst → docs/source/benchmarking/performance.rst b/docs/source/performance.rst → docs/source/benchmarking/performance.rst
@@ -131,7 +131,7 @@ To use Optimizer Sharded Training, refer to :ref:`model-parallelism`.
 
 Sharded DDP can work across all DDP variants by adding the additional ``--plugins ddp_sharded`` flag.
 
-Refer to the :ref:`distributed computing guide for more details <multi_gpu>`.
+Refer to the :doc:`distributed computing guide for more details <../advanced/multi_gpu>`.
 
 
 Sequential Model Parallelism with Checkpointing

diff --git a/docs/source/cloud_training.rst → docs/source/clouds/cloud_training.rst b/docs/source/cloud_training.rst → docs/source/clouds/cloud_training.rst
diff --git a/docs/source/slurm.rst → docs/source/clouds/slurm.rst b/docs/source/slurm.rst → docs/source/clouds/slurm.rst
@@ -17,7 +17,7 @@ Multi-node training
 -------------------
 To train a model using multiple nodes, do the following:
 
-1.  Design your :ref:`lightning_module`.
+1.  Design your :doc:`lightning module <../common/lightning_module>`.
 
 2.  Enable DDP in the trainer
 

diff --git a/docs/source/child_modules.rst → docs/source/common/child_modules.rst b/docs/source/child_modules.rst → docs/source/common/child_modules.rst
diff --git a/docs/source/debugging.rst → docs/source/common/debugging.rst b/docs/source/debugging.rst → docs/source/common/debugging.rst
diff --git a/docs/source/early_stopping.rst → docs/source/common/early_stopping.rst b/docs/source/early_stopping.rst → docs/source/common/early_stopping.rst
diff --git a/docs/source/fast_training.rst → docs/source/common/fast_training.rst b/docs/source/fast_training.rst → docs/source/common/fast_training.rst
diff --git a/docs/source/hyperparameters.rst → docs/source/common/hyperparameters.rst b/docs/source/hyperparameters.rst → docs/source/common/hyperparameters.rst
diff --git a/docs/source/lightning_module.rst → docs/source/common/lightning_module.rst b/docs/source/lightning_module.rst → docs/source/common/lightning_module.rst
diff --git a/docs/source/loggers.rst → docs/source/common/loggers.rst b/docs/source/loggers.rst → docs/source/common/loggers.rst
@@ -17,7 +17,7 @@ but you can pass to the :class:`~pytorch_lightning.trainer.trainer.Trainer` any
     All loggers log by default to `os.getcwd()`. To change the path without creating a logger set
     `Trainer(default_root_dir='/your/path/to/save/checkpoints')`
 
-Read more about :ref:`logging` options.
+Read more about :doc:`logging <../extensions/logging>` options.
 
 To log arbitrary artifacts like images or audio samples use the `trainer.log_dir` property to resolve
 the path.

diff --git a/docs/source/optimizers.rst → docs/source/common/optimizers.rst b/docs/source/optimizers.rst → docs/source/common/optimizers.rst
diff --git a/docs/source/production_inference.rst → docs/source/common/production_inference.rst b/docs/source/production_inference.rst → docs/source/common/production_inference.rst
diff --git a/docs/source/single_gpu.rst → docs/source/common/single_gpu.rst b/docs/source/single_gpu.rst → docs/source/common/single_gpu.rst
diff --git a/docs/source/test_set.rst → docs/source/common/test_set.rst b/docs/source/test_set.rst → docs/source/common/test_set.rst
@@ -74,7 +74,7 @@ running the test set (ie: 16-bit, dp, ddp, etc...)
 Test with additional data loaders
 ---------------------------------
 You can still run inference on a test set even if the `test_dataloader` method hasn't been
-defined within your :ref:`lightning_module` instance. This would be the case when your test data
+defined within your :doc:`lightning module <../common/lightning_module>` instance. This would be the case when your test data
 is not available at the time your model was declared.
 
 .. code-block:: python
@@ -87,7 +87,7 @@ is not available at the time your model was declared.
 
 You can either pass in a single dataloader or a list of them. This optional named
 parameter can be used in conjunction with any of the above use cases. Additionally,
-you can also pass in an :ref:`datamodules` that have overridden the
+you can also pass in an :doc:`datamodules <../extensions/datamodules>` that have overridden the
 :ref:`datamodule-test-dataloader-label` method.
 
 .. code-block:: python

diff --git a/docs/source/trainer.rst → docs/source/common/trainer.rst b/docs/source/trainer.rst → docs/source/common/trainer.rst
@@ -454,7 +454,7 @@ Example::
     trainer.tune(model)
 
 .. note::
-    See the :ref:`learning rate finder guide <lr_finder>`.
+    See the :doc:`learning rate finder guide <../advanced/lr_finder>`.
 
 benchmark
 ^^^^^^^^^
@@ -580,7 +580,7 @@ To disable automatic checkpointing, set this to `False`.
 
 You can override the default behavior by initializing the :class:`~pytorch_lightning.callbacks.ModelCheckpoint`
 callback, and adding it to the :paramref:`~pytorch_lightning.trainer.trainer.Trainer.callbacks` list.
-See :ref:`Saving and Loading Weights <weights_loading>` for how to customize checkpointing.
+See :doc:`Saving and Loading Weights <../common/weights_loading>` for how to customize checkpointing.
 
 .. testcode::
 
@@ -691,7 +691,7 @@ Writes logs to disk this often.
     trainer = Trainer(flush_logs_every_n_steps=100)
 
 See Also:
-    - :ref:`logging`
+    - :doc:`logging <../extensions/logging>`
 
 gpus
 ^^^^
@@ -737,7 +737,7 @@ Example::
     trainer = Trainer(gpus=[1, 4], num_nodes=4)
 
 See Also:
-    - :ref:`Multi-GPU training guide <multi_gpu>`.
+    - :doc:`Multi-GPU training guide <../advanced/multi_gpu>`.
 
 gradient_clip_val
 ^^^^^^^^^^^^^^^^^
@@ -862,7 +862,7 @@ How often to add logging rows (does not write to disk)
     trainer = Trainer(log_every_n_steps=50)
 
 See Also:
-    - :ref:`logging`
+    - :doc:`logging <../extensions/logging>`
 
 log_gpu_memory
 ^^^^^^^^^^^^^^
@@ -905,7 +905,7 @@ logger
 
 |
 
-:ref:`Logger <loggers>` (or iterable collection of loggers) for experiment tracking.
+:doc:`Logger <../common/loggers>` (or iterable collection of loggers) for experiment tracking.
 
 .. testcode::
 
@@ -1222,7 +1222,7 @@ profiler
 
 To profile individual steps during training and assist in identifying bottlenecks.
 
-See the :ref:`profiler documentation <profiler>`. for more details.
+See the :doc:`profiler documentation <../advanced/profiler>`. for more details.
 
 .. testcode::
 

diff --git a/docs/source/weights_loading.rst → docs/source/common/weights_loading.rst b/docs/source/weights_loading.rst → docs/source/common/weights_loading.rst
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -25,6 +25,7 @@
 PATH_ROOT = os.path.join(PATH_HERE, '..', '..')
 sys.path.insert(0, os.path.abspath(PATH_ROOT))
 
+FOLDER_GENERATED = 'generated'
 SPHINX_MOCK_REQUIREMENTS = int(os.environ.get('SPHINX_MOCK_REQUIREMENTS', True))
 if SPHINX_MOCK_REQUIREMENTS:
     builtins.__LIGHTNING_SETUP__ = True
@@ -49,22 +50,30 @@
 # with open('readme.md', 'w') as fp:
 #     fp.write(readme)
 
+
+def _transform_changelog(path_in: str, path_out: str) -> None:
+    with open(path_in, 'r') as fp:
+        chlog_lines = fp.readlines()
+    # enrich short subsub-titles to be unique
+    chlog_ver = ''
+    for i, ln in enumerate(chlog_lines):
+        if ln.startswith('## '):
+            chlog_ver = ln[2:].split('-')[0].strip()
+        elif ln.startswith('### '):
+            ln = ln.replace('###', f'### {chlog_ver} -')
+            chlog_lines[i] = ln
+    with open(path_out, 'w') as fp:
+        fp.writelines(chlog_lines)
+
+
+os.makedirs(os.path.join(PATH_HERE, FOLDER_GENERATED), exist_ok=True)
 # copy all documents from GH templates like contribution guide
 for md in glob.glob(os.path.join(PATH_ROOT, '.github', '*.md')):
-    shutil.copy(md, os.path.join(PATH_HERE, os.path.basename(md)))
+    shutil.copy(md, os.path.join(PATH_HERE, FOLDER_GENERATED, os.path.basename(md)))
 # copy also the changelog
-with open(os.path.join(PATH_ROOT, 'CHANGELOG.md'), 'r') as fp:
-    chlog_lines = fp.readlines()
-# enrich short subsub-titles to be unique
-chlog_ver = ''
-for i, ln in enumerate(chlog_lines):
-    if ln.startswith('## '):
-        chlog_ver = ln[2:].split('-')[0].strip()
-    elif ln.startswith('### '):
-        ln = ln.replace('###', f'### {chlog_ver} -')
-        chlog_lines[i] = ln
-with open(os.path.join(PATH_HERE, 'CHANGELOG.md'), 'w') as fp:
-    fp.writelines(chlog_lines)
+_transform_changelog(
+    os.path.join(PATH_ROOT, 'CHANGELOG.md'), os.path.join(PATH_HERE, FOLDER_GENERATED, 'CHANGELOG.md')
+)
 
 # -- Project information -----------------------------------------------------
 
@@ -146,7 +155,7 @@
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = [
-    'PULL_REQUEST_TEMPLATE.md',
+    f'{FOLDER_GENERATED}/PULL_REQUEST_TEMPLATE.md',
 ]
 
 # The name of the Pygments (syntax highlighting) style to use.

diff --git a/docs/source/asr_nlp_tts.rst → docs/source/ecosystem/asr_nlp_tts.rst b/docs/source/asr_nlp_tts.rst → docs/source/ecosystem/asr_nlp_tts.rst
diff --git a/docs/source/bolts.rst → docs/source/ecosystem/bolts.rst b/docs/source/bolts.rst → docs/source/ecosystem/bolts.rst
diff --git a/docs/source/community_examples.rst → docs/source/ecosystem/community_examples.rst b/docs/source/community_examples.rst → docs/source/ecosystem/community_examples.rst
diff --git a/docs/source/pytorch_ecoystem.rst → docs/source/ecosystem/pytorch_ecoystem.rst b/docs/source/pytorch_ecoystem.rst → docs/source/ecosystem/pytorch_ecoystem.rst
diff --git a/docs/source/accelerators.rst → docs/source/extensions/accelerators.rst b/docs/source/accelerators.rst → docs/source/extensions/accelerators.rst
diff --git a/docs/source/callbacks.rst → docs/source/extensions/callbacks.rst b/docs/source/callbacks.rst → docs/source/extensions/callbacks.rst
@@ -22,7 +22,7 @@ Callback
 A callback is a self-contained program that can be reused across projects.
 
 Lightning has a callback system to execute callbacks when needed. Callbacks should capture NON-ESSENTIAL
-logic that is NOT required for your :ref:`lightning_module` to run.
+logic that is NOT required for your :doc:`lightning module <../common/lightning_module>` to run.
 
 Here's the flow of how the callback hooks are executed:
 
@@ -63,7 +63,7 @@ Example:
     trainer is init now
 
 We successfully extended functionality without polluting our super clean
-:ref:`lightning_module` research code.
+:doc:`lightning module <../common/lightning_module>` research code.
 
 -----------
 

diff --git a/docs/source/datamodules.rst → docs/source/extensions/datamodules.rst b/docs/source/datamodules.rst → docs/source/extensions/datamodules.rst
diff --git a/docs/source/logging.rst → docs/source/extensions/logging.rst b/docs/source/logging.rst → docs/source/extensions/logging.rst
@@ -60,7 +60,7 @@ Lightning offers automatic log functionalities for logging scalars, or manual lo
 Automatic Logging
 =================
 Use the :func:`~~pytorch_lightning.core.lightning.LightningModule.log`
-method to log from anywhere in a :ref:`lightning_module` and :ref:`callbacks`
+method to log from anywhere in a :doc:`lightning module <../common/lightning_module>` and :doc:`callbacks <../extensions/callbacks>`
 except functions with `batch_start` in their names.
 
 .. code-block:: python
@@ -90,7 +90,7 @@ The :func:`~~pytorch_lightning.core.lightning.LightningModule.log` method has a
 .. note::
 
     -   Setting ``on_epoch=True`` will cache all your logged values during the full training epoch and perform a
-        reduction `on_epoch_end`. We recommend using the :ref:`metrics` API when working with custom reduction.
+        reduction `on_epoch_end`. We recommend using the :doc:`metrics <../extensions/metrics>` API when working with custom reduction.
 
     -   Setting both ``on_step=True`` and ``on_epoch=True`` will create two keys per metric you log with
         suffix ``_step`` and ``_epoch``, respectively. You can refer to these keys e.g. in the `monitor`

diff --git a/docs/source/metrics.rst → docs/source/extensions/metrics.rst b/docs/source/metrics.rst → docs/source/extensions/metrics.rst
diff --git a/docs/source/plugins.rst → docs/source/extensions/plugins.rst b/docs/source/plugins.rst → docs/source/extensions/plugins.rst