Doc pipelines

docs/source/index.rst

@@ -207,3 +207,4 @@ conversion utilities for the following models:
     model_doc/dpr
     internal/modeling_utils
     internal/tokenization_utils
+    internal/pipelines_utils

docs/source/internal/pipelines_utils.rst

@@ -0,0 +1,40 @@
+Utilities for pipelines
+-----------------------
+
+This page lists all the utility functions the library provides for pipelines.
+
+Most of those are only useful if you are studying the code of the models in the library.
+
+
+Argument handling
+~~~~~~~~~~~~~~~~~
+
+.. autoclass:: transformers.pipelines.ArgumentHandler
+
+.. autoclass:: transformers.pipelines.ZeroShotClassificationArgumentHandler
+
+.. autoclass:: transformers.pipelines.QuestionAnsweringArgumentHandler
+
+
+Data format
+~~~~~~~~~~~
+
+.. autoclass:: transformers.pipelines.PipelineDataFormat
+    :members:
+
+.. autoclass:: transformers.pipelines.CsvPipelineDataFormat
+    :members:
+
+.. autoclass:: transformers.pipelines.JsonPipelineDataFormat
+    :members:
+
+.. autoclass:: transformers.pipelines.PipedPipelineDataFormat
+    :members:
+
+
+Utilities
+~~~~~~~~~
+
+.. autofunction:: transformers.pipelines.get_framework
+
+.. autoclass:: transformers.pipelines.PipelineException

docs/source/main_classes/model.rst

@@ -41,3 +41,9 @@ The other methods that are common to each model are defined in :class:`~transfor
 
 .. autoclass:: transformers.modeling_tf_utils.TFModelUtilsMixin
     :members:
+
+
+Generative models
+~~~~~~~~~~~~~~~~~
+
+Coming soon

docs/source/main_classes/pipelines.rst

@@ -3,13 +3,23 @@ Pipelines
 
 The pipelines are a great and easy way to use models for inference. These pipelines are objects that abstract most
 of the complex code from the library, offering a simple API dedicated to several tasks, including Named Entity
-Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering.
+Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering. See the
+:doc:`task summary <../task_summary>` for examples of use.
 
 There are two categories of pipeline abstractions to be aware about:
 
-- The :func:`~transformers.pipeline` which is the most powerful object encapsulating all other pipelines
-- The other task-specific pipelines, such as :class:`~transformers.TokenClassificationPipeline`
-  or :class:`~transformers.QuestionAnsweringPipeline`
+- The :func:`~transformers.pipeline` which is the most powerful object encapsulating all other pipelines.
+- The other task-specific pipelines:
+
+    - :class:`~transformers.ConversationalPipeline`
+    - :class:`~transformers.FeatureExtractionPipeline`
+    - :class:`~transformers.FillMaskPipeline`
+    - :class:`~transformers.QuestionAnsweringPipeline`
+    - :class:`~transformers.SummarizationPipeline`
+    - :class:`~transformers.TextClassificationPipeline`
+    - :class:`~transformers.TextGenerationPipeline`
+    - :class:`~transformers.TokenClassificationPipeline`
+    - :class:`~transformers.TranslationPipeline`
 
 The pipeline abstraction
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -21,61 +31,75 @@ other pipeline but requires an additional argument which is the `task`.
 
 
 The task specific pipelines
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Parent class: Pipeline
-=========================================
-
-.. autoclass:: transformers.Pipeline
-    :members: predict, transform, save_pretrained
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-TokenClassificationPipeline
+ConversationalPipeline
 ==========================================
 
-.. autoclass:: transformers.TokenClassificationPipeline
+.. autoclass:: transformers.Conversation
 
-NerPipeline
+.. autoclass:: transformers.ConversationalPipeline
+    :special-members: __call__
+    :members:
+
+FeatureExtractionPipeline
 ==========================================
 
-This class is an alias of the :class:`~transformers.TokenClassificationPipeline` defined above. Please refer to that pipeline for
-documentation and usage examples.
+.. autoclass:: transformers.FeatureExtractionPipeline
+    :special-members: __call__
+    :members:
 
 FillMaskPipeline
 ==========================================
 
 .. autoclass:: transformers.FillMaskPipeline
+    :special-members: __call__
+    :members:
 
-FeatureExtractionPipeline
-==========================================
-
-.. autoclass:: transformers.FeatureExtractionPipeline
-
-TextClassificationPipeline
+NerPipeline
 ==========================================
 
-.. autoclass:: transformers.TextClassificationPipeline
+This class is an alias of the :class:`~transformers.TokenClassificationPipeline` defined below. Please refer to that
+pipeline for documentation and usage examples.
 
 QuestionAnsweringPipeline
 ==========================================
 
 .. autoclass:: transformers.QuestionAnsweringPipeline
-
+    :special-members: __call__
+    :members:
 
 SummarizationPipeline
 ==========================================
 
 .. autoclass:: transformers.SummarizationPipeline
+    :special-members: __call__
+    :members:
 
+TextClassificationPipeline
+==========================================
+
+.. autoclass:: transformers.TextClassificationPipeline
+    :special-members: __call__
+    :members:
 
 TextGenerationPipeline
 ==========================================
 
 .. autoclass:: transformers.TextGenerationPipeline
+    :special-members: __call__
+    :members:
 
-
-ConversationalPipeline
+TokenClassificationPipeline
 ==========================================
 
-.. autoclass:: transformers.Conversation
+.. autoclass:: transformers.TokenClassificationPipeline
+    :special-members: __call__
+    :members:
+
 
-.. autoclass:: transformers.ConversationalPipeline
+Parent class: :obj:`Pipeline`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. autoclass:: transformers.Pipeline
+    :members: