Neural Graphs

docs/docs_zh/sources/source/tutorials/module_configuration.rst

@@ -21,14 +21,14 @@
 
 .. literalinclude:: ../../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 25-35
+   :lines: 24-34
 
 现在我们可以导出任何一个已有模块的配置，调用 :meth:`export_to_config()`, 例如 \
 我们可以导出 :class:`TaylorNet` 的配置，通过调用:
 
 .. literalinclude:: ../../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 38
+   :lines: 37
 
 导入配置
 ---------
@@ -37,7 +37,7 @@
 
 .. literalinclude:: ../../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 41
+   :lines: 40
 
 .. note::
     :meth:`import_from_config()` 函数事实上是创建了在配置中的这个类的一个新的实例 \
@@ -49,7 +49,7 @@
 
 .. literalinclude:: ../../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 43-
+   :lines: 42-
 
 
 .. include:: module_custom_configuration.rst

docs/docs_zh/sources/source/tutorials/module_custom_configuration.rst

@@ -15,21 +15,21 @@
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 33-35
+   :lines: 28-30
 
 现在让我们定义 :class:`CustomTaylorNet` 神经模块类:
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 38-43
+   :lines: 33-38
 
 
 为了能处理好 :class:`Status` enum 的导出功能，我们必须实现自定义函数 \
 :meth:`export_to_config()`:
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 45-76
+   :lines: 40-61
 
 
 注意配置实际上是一个字典，包含了两个部分:
@@ -40,35 +40,28 @@
 这些参数存在保护域 ``self._init_params`` 中，它的基类是 :class:`NeuralModule` 类。
 确保用户不能直接访问和使用它们。 
 
-类似地，我们必须重载方法 :meth:`import_from_config()` :
+类似地，我们必须重载方法 :meth:`_deserialize_configuration()` :
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 79-119
-
-请注意，基类 :class:`NeuralModule` 提供了一些保护方法供我们使用, \
-其中，最重要的是:
-
- * :meth:`_create_config_header()` 生成合适的 header, 以及 \
- * :meth:`_validate_config_file()` 验证加载的配置文件 (检查 header 内容)。
-
+   :lines: 63-86
 
 .. note::
-    再强调一下 :meth:`import_from_config()` 是类的方法，实际上返回 \
+    再强调一下 :meth:`_deserialize_configuration()` 是类的方法，实际上返回 \
     一个新的对象实例 - 在这个例子中就是 :class:`CustomTaylorNet` 类型。
 
 
 现在我们可以简单的构建一个实例，并且导出它的配置，通过调用:
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 128-129,134-135
+   :lines: 95-96,101-102
 
 通过加载这个配置，初始化第二个实例:
 
 .. literalinclude:: ../../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 137-139
+   :lines: 104-106
 
 从结果中我们可以看到新的对象把状态都设置成了原来那个对象的值:
 

docs/sources/source/tutorials/module_configuration.rst

@@ -17,18 +17,18 @@ In the following example we will once again train a model to learn Taylor's coef
 However, we will extend the example by showing how to export configuration of the module to a YAML file and \
 create a second instance having the same set of parameters.
 
-Let us start by creating the :class:`NeuralFactory` object and instatiating the modules from the original example:
+Let us start by creating the :class:`NeuralModuleFactory` object and instatiating the modules from the original example:
 
 .. literalinclude:: ../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 25-35
+   :lines: 24-34
 
 Now we can export the configuration of any of the existing modules by using  the :meth:`export_to_config()`, for \
 example we can export the configuration of the trainable :class:`TaylorNet` by calling:
 
 .. literalinclude:: ../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 38
+   :lines: 37
 
 Importing the configuration
 ---------------------------
@@ -37,7 +37,7 @@ There is an analogical function :meth:`import_from_config()` responsible for loa
 
 .. literalinclude:: ../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 41
+   :lines: 40
 
 .. note::
     The :meth:`import_from_config()` function actually creates a new instance of object of the class that was stored \
@@ -49,7 +49,7 @@ For example, we can build a graph and train it with a NeMo trainer:
 
 .. literalinclude:: ../../../../examples/start_here/module_configuration.py
    :language: python
-   :lines: 43-
+   :lines: 42-
 
 
 .. include:: module_custom_configuration.rst

docs/sources/source/tutorials/module_custom_configuration.rst

@@ -15,21 +15,21 @@ and extend it by those methods. But first, let us define a simple :class:`Status
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 33-35
+   :lines: 28-30
 
 Now let us define the :class:`CustomTaylorNet` Neural Module class:
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 38-43
+   :lines: 33-38
 
 
 In order to properly handle the export of the :class:`Status` enum we must implement a custom function \
-:meth:`export_to_config()`:
+:meth:`_serialize_configuration()`:
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 45-76
+   :lines: 49-61
 
 
 Note that the configuration is actually a dictionary consisting of two sections:
@@ -40,35 +40,29 @@ Note that the configuration is actually a dictionary consisting of two sections:
 Those parameters are stored in the protected ``self._init_params``  field of the base :class:`NeuralModule` class.
 It is assumed that (aside of this use-case) the user won't access nor use them directly.
 
-Analogically, we must overload the :meth:`import_from_config()` method:
+Analogically, we must overload the :meth:`_deserialize_configuration()` method:
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 79-119
-
-Please note that the base :class:`NeuralModule` class provides several protected methods that we used, \
-with most important being:
-
- * :meth:`_create_config_header()` generating the appropriate header, and \
- * :meth:`_validate_config_file()` validating the loaded configuration file (checking the header content).
-
+   :lines: 63-86
 
 .. note::
-    It is once again worth emphasizing that the :meth:`import_from_config()` is a class method, actually returning a \
-    new object instance - in this case of the hardcoded :class:`CustomTaylorNet` type.
+    It is worth emphasizing that the :meth:`_deserialize_configuration()` is a class method, 
+    analogically to public :meth:`import_from_config()` and :meth:`deserialize()` methods
+    that return a new object instance - in this case of the hardcoded :class:`CustomTaylorNet` type.
 
 
 Now we can simply create an instance and export its configuration by calling:
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 128-129,134-135
+   :lines: 95-96,101-102
 
 And instantiate a second by loading that configuration:
 
 .. literalinclude:: ../../../../examples/start_here/module_custom_configuration.py
    :language: python
-   :lines: 137-139
+   :lines: 104-106
 
 As a result we will see that the new object has set the status to the same value as the original one:
 

examples/start_here/graph_composition_integration_tests0_jasper.py

@@ -0,0 +1,122 @@
+# ! /usr/bin/python
+# -*- coding: utf-8 -*-
+
+# =============================================================================
+# Copyright (c) 2020 NVIDIA. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# =============================================================================
+
+from functools import partial
+from os.path import expanduser
+
+from ruamel.yaml import YAML
+
+import nemo
+import nemo.collections.asr as nemo_asr
+from nemo.collections.asr.helpers import monitor_asr_train_progress
+from nemo.core import NeuralGraph, OperationMode
+from nemo.utils import logging
+
+nf = nemo.core.NeuralModuleFactory()
+
+logging.info(
+    "This example shows how one can build a Jasper model using the explicit graph."
+    F" This approach works for applications containing a single graph."
+)
+
+# Set paths to "manifests" and model configuration files.
+train_manifest = "~/TestData/an4_dataset/an4_train.json"
+val_manifest = "~/TestData/an4_dataset/an4_val.json"
+model_config_file = "~/workspace/nemo/examples/asr/configs/jasper_an4.yaml"
+
+yaml = YAML(typ="safe")
+with open(expanduser(model_config_file)) as f:
+    config = yaml.load(f)
+# Get vocabulary.
+vocab = config['labels']
+
+# Create neural modules.
+data_layer = nemo_asr.AudioToTextDataLayer.deserialize(
+    config["AudioToTextDataLayer_train"], overwrite_params={"manifest_filepath": train_manifest, "batch_size": 16},
+)
+
+data_preprocessor = nemo_asr.AudioToMelSpectrogramPreprocessor.deserialize(config["AudioToMelSpectrogramPreprocessor"])
+
+jasper_encoder = nemo_asr.JasperEncoder.deserialize(config["JasperEncoder"])
+jasper_decoder = nemo_asr.JasperDecoderForCTC.deserialize(
+    config["JasperDecoderForCTC"], overwrite_params={"num_classes": len(vocab)}
+)
+ctc_loss = nemo_asr.CTCLossNM(num_classes=len(vocab))
+greedy_decoder = nemo_asr.GreedyCTCDecoder()
+
+# Create the Jasper "model".
+with NeuralGraph(operation_mode=OperationMode.both) as Jasper:
+    # Copy one input port definitions - using "user" port names.
+    Jasper.inputs["input"] = data_preprocessor.input_ports["input_signal"]
+    # Bind selected inputs - bind other using the default port name.
+    i_processed_signal, i_processed_signal_len = data_preprocessor(input_signal=Jasper.inputs["input"], length=Jasper)
+    i_encoded, i_encoded_len = jasper_encoder(audio_signal=i_processed_signal, length=i_processed_signal_len)
+    i_log_probs = jasper_decoder(encoder_output=i_encoded)
+    # Bind selected outputs - using "user" port names.
+    Jasper.outputs["log_probs"] = i_log_probs
+    Jasper.outputs["encoded_len"] = i_encoded_len
+
+# Print the summary.
+logging.info(Jasper.summary())
+
+# Serialize graph
+serialized_jasper = Jasper.serialize()
+print("Serialized:\n", serialized_jasper)
+
+# Delete everything - aside of jasper encoder, just as a test to show that reusing work! ;)
+del Jasper
+del data_preprocessor
+# del jasper_encoder #
+del jasper_decoder
+
+# Deserialize graph - copy of the JASPER "model".
+jasper_copy = NeuralGraph.deserialize(serialized_jasper, reuse_existing_modules=True, name="jasper_copy")
+serialized_jasper_copy = jasper_copy.serialize()
+# print("Deserialized:\n", serialized_jasper_copy)
+assert serialized_jasper == serialized_jasper_copy
+
+# Create the "training" graph.
+with NeuralGraph(name="training") as training_graph:
+    # Create the "implicit" training graph.
+    o_audio_signal, o_audio_signal_len, o_transcript, o_transcript_len = data_layer()
+    # Use Jasper module as any other neural module.
+    o_log_probs, o_encoded_len = jasper_copy(input=o_audio_signal, length=o_audio_signal_len)
+    o_predictions = greedy_decoder(log_probs=o_log_probs)
+    o_loss = ctc_loss(
+        log_probs=o_log_probs, targets=o_transcript, input_length=o_encoded_len, target_length=o_transcript_len
+    )
+    # Set graph output.
+    training_graph.outputs["o_loss"] = o_loss
+    # training_graph.outputs["o_predictions"] = o_predictions # DOESN'T WORK?!?
+
+# Print the summary.
+logging.info(training_graph.summary())
+
+tensors_to_evaluate = [o_loss, o_predictions, o_transcript, o_transcript_len]
+train_callback = nemo.core.SimpleLossLoggerCallback(
+    tensors=tensors_to_evaluate, print_func=partial(monitor_asr_train_progress, labels=vocab)
+)
+# import pdb;pdb.set_trace()
+nf.train(
+    # tensors_to_optimize=[o_loss, o_predictions], # DOESN'T WORK?!?
+    training_graph=training_graph,
+    optimizer="novograd",
+    callbacks=[train_callback],
+    optimization_params={"num_epochs": 50, "lr": 0.01},
+)

examples/start_here/graph_composition_integration_tests1.py

@@ -0,0 +1,48 @@
+# ! /usr/bin/python
+# -*- coding: utf-8 -*-
+
+# =============================================================================
+# Copyright (c) 2020 NVIDIA. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# =============================================================================
+
+from nemo.backends.pytorch.tutorials import MSELoss, RealFunctionDataLayer, TaylorNet
+from nemo.core import DeviceType, NeuralGraph, NeuralModuleFactory, OperationMode, SimpleLossLoggerCallback
+from nemo.utils import logging
+
+nf = NeuralModuleFactory(placement=DeviceType.CPU)
+# Instantiate the necessary neural modules.
+dl = RealFunctionDataLayer(n=100, batch_size=32)
+m2 = TaylorNet(dim=4)
+loss = MSELoss()
+
+logging.info("This example shows how one can build an `explicit` graph.")
+
+with NeuralGraph(operation_mode=OperationMode.training) as g0:
+    x, t = dl()
+    p = m2(x=x)
+    lss = loss(predictions=p, target=t)
+    # Manual bind.
+    g0.outputs["output"] = lss
+
+# Print the summary.
+logging.info(g0.summary())
+
+# SimpleLossLoggerCallback will print loss values to console.
+callback = SimpleLossLoggerCallback(
+    tensors=[lss], print_func=lambda x: logging.info(f'Train Loss: {str(x[0].item())}'),
+)
+
+# Invoke "train" action.
+nf.train([lss], callbacks=[callback], optimization_params={"num_epochs": 2, "lr": 0.0003}, optimizer="sgd")