Upgrade pydoc-markdown

.github/workflows/update_docsstrings_tutorials.yml

@@ -28,7 +28,7 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install pydoc-markdown==3.11.0
+          pip install pydoc-markdown
           pip install mkdocs
           pip install jupytercontrib
           pip install watchdog==1.0.2

docs/_src/api/api/generator.md

@@ -1,7 +1,9 @@
-<a name="base"></a>
+<a id="base"></a>
+
 # Module base
 
-<a name="base.BaseGenerator"></a>
+<a id="base.BaseGenerator"></a>
+
 ## BaseGenerator
 
 ```python
@@ -10,30 +12,28 @@ class BaseGenerator(BaseComponent)
 
 Abstract class for Generators
 
-<a name="base.BaseGenerator.predict"></a>
+<a id="base.BaseGenerator.predict"></a>
+
 #### predict
 
 ```python
- | @abstractmethod
- | predict(query: str, documents: List[Document], top_k: Optional[int]) -> Dict
+@abstractmethod
+def predict(query: str, documents: List[Document], top_k: Optional[int]) -> Dict
 ```
 
 Abstract method to generate answers.
 
-**Arguments**:
-
-- `query`: Query
-- `documents`: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
-- `top_k`: Number of returned answers
-
-**Returns**:
+:param query: Query
+:param documents: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
+:param top_k: Number of returned answers
+:return: Generated answers plus additional infos in a dict
 
-Generated answers plus additional infos in a dict
+<a id="transformers"></a>
 
-<a name="transformers"></a>
 # Module transformers
 
-<a name="transformers.RAGenerator"></a>
+<a id="transformers.RAGenerator"></a>
+
 ## RAGenerator
 
 ```python
@@ -80,51 +80,21 @@ i.e. the model can easily adjust to domain documents even after training has fin
 |      }}]}
 ```
 
-<a name="transformers.RAGenerator.__init__"></a>
-#### \_\_init\_\_
+<a id="transformers.RAGenerator.predict"></a>
 
-```python
- | __init__(model_name_or_path: str = "facebook/rag-token-nq", model_version: Optional[str] = None, retriever: Optional[DensePassageRetriever] = None, generator_type: RAGeneratorType = RAGeneratorType.TOKEN, top_k: int = 2, max_length: int = 200, min_length: int = 2, num_beams: int = 2, embed_title: bool = True, prefix: Optional[str] = None, use_gpu: bool = True)
-```
-
-Load a RAG model from Transformers along with passage_embedding_model.
-See https://huggingface.co/transformers/model_doc/rag.html for more details
-
-**Arguments**:
-
-- `model_name_or_path`: Directory of a saved model or the name of a public model e.g.
-                           'facebook/rag-token-nq', 'facebook/rag-sequence-nq'.
-                           See https://huggingface.co/models for full list of available models.
-- `model_version`: The version of model to use from the HuggingFace model hub. Can be tag name, branch name, or commit hash.
-- `retriever`: `DensePassageRetriever` used to embedded passages for the docs passed to `predict()`. This is optional and is only needed if the docs you pass don't already contain embeddings in `Document.embedding`.
-- `generator_type`: Which RAG generator implementation to use? RAG-TOKEN or RAG-SEQUENCE
-- `top_k`: Number of independently generated text to return
-- `max_length`: Maximum length of generated text
-- `min_length`: Minimum length of generated text
-- `num_beams`: Number of beams for beam search. 1 means no beam search.
-- `embed_title`: Embedded the title of passage while generating embedding
-- `prefix`: The prefix used by the generator's tokenizer.
-- `use_gpu`: Whether to use GPU. Falls back on CPU if no GPU is available.
-
-<a name="transformers.RAGenerator.predict"></a>
 #### predict
 
 ```python
- | predict(query: str, documents: List[Document], top_k: Optional[int] = None) -> Dict
+def predict(query: str, documents: List[Document], top_k: Optional[int] = None) -> Dict
 ```
 
 Generate the answer to the input query. The generation will be conditioned on the supplied documents.
 These document can for example be retrieved via the Retriever.
 
-**Arguments**:
-
-- `query`: Query
-- `documents`: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
-- `top_k`: Number of returned answers
-
-**Returns**:
-
-Generated answers plus additional infos in a dict like this:
+:param query: Query
+:param documents: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
+:param top_k: Number of returned answers
+:return: Generated answers plus additional infos in a dict like this:
 
 ```python
 |     {'query': 'who got the first nobel prize in physics',
@@ -139,7 +109,8 @@ Generated answers plus additional infos in a dict like this:
 |      }}]}
 ```
 
-<a name="transformers.Seq2SeqGenerator"></a>
+<a id="transformers.Seq2SeqGenerator"></a>
+
 ## Seq2SeqGenerator
 
 ```python
@@ -189,44 +160,19 @@ For a list of all text-generation models see https://huggingface.co/models?pipel
 |
 ```
 
-<a name="transformers.Seq2SeqGenerator.__init__"></a>
-#### \_\_init\_\_
+<a id="transformers.Seq2SeqGenerator.predict"></a>
 
-```python
- | __init__(model_name_or_path: str, input_converter: Optional[Callable] = None, top_k: int = 1, max_length: int = 200, min_length: int = 2, num_beams: int = 8, use_gpu: bool = True)
-```
-
-**Arguments**:
-
-- `model_name_or_path`: a HF model name for auto-regressive language model like GPT2, XLNet, XLM, Bart, T5 etc
-- `input_converter`: an optional Callable to prepare model input for the underlying language model
-                        specified in model_name_or_path parameter. The required __call__ method signature for
-                        the Callable is:
-                        __call__(tokenizer: PreTrainedTokenizer, query: str, documents: List[Document],
-                        top_k: Optional[int] = None) -> BatchEncoding:
-- `top_k`: Number of independently generated text to return
-- `max_length`: Maximum length of generated text
-- `min_length`: Minimum length of generated text
-- `num_beams`: Number of beams for beam search. 1 means no beam search.
-- `use_gpu`: Whether to use GPU or the CPU. Falls back on CPU if no GPU is available.
-
-<a name="transformers.Seq2SeqGenerator.predict"></a>
 #### predict
 
 ```python
- | predict(query: str, documents: List[Document], top_k: Optional[int] = None) -> Dict
+def predict(query: str, documents: List[Document], top_k: Optional[int] = None) -> Dict
 ```
 
 Generate the answer to the input query. The generation will be conditioned on the supplied documents.
 These document can be retrieved via the Retriever or supplied directly via predict method.
 
-**Arguments**:
-
-- `query`: Query
-- `documents`: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
-- `top_k`: Number of returned answers
-
-**Returns**:
-
-Generated answers
+:param query: Query
+:param documents: Related documents (e.g. coming from a retriever) that the answer shall be conditioned on.
+:param top_k: Number of returned answers
+:return: Generated answers
 

docs/_src/api/api/pydoc-markdown-answer-generator.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/answer_generator]
     modules: ['base', 'transformers']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-crawler.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/connector]
     modules: ['crawler']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-document-classifier.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/document_classifier]
     modules: ['base', 'transformers']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-document-store.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/document_stores]
     modules: ['base', 'elasticsearch', 'memory', 'sql', 'faiss', 'milvus', 'weaviate', 'graphdb', 'deepsetcloud']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-evaluation.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/evaluator]
     modules: ['evaluator']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-extractor.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/extractor]
     modules: ['entity']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-file-classifier.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/file_classifier]
     modules: ['file_type']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-file-converters.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/file_converter]
     modules: ['base', 'docx', 'image', 'markdown', 'pdf', 'tika', 'txt']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-other.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/other]
     modules: ['docs2answers', 'join_docs']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-pipelines.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/pipelines]
     modules: ['base', 'standard_pipelines']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-preprocessor.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/nodes/preprocessor]
     modules: ['base', 'preprocessor']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false

docs/_src/api/api/pydoc-markdown-primitives.yml

@@ -3,12 +3,12 @@ loaders:
     search_path: [../../../../haystack/]
     modules: ['schema']
     ignore_when_discovered: ['__init__']
-processor:
+processors:
   - type: filter
     expression: not name.startswith('_') and default()
-  - documented_only: true
-  - do_not_filter_modules: false
-  - skip_empty_modules: true
+    documented_only: true
+    do_not_filter_modules: false
+    skip_empty_modules: true
 renderer:
   type: markdown
   descriptive_class_title: false