Fix Inc Build (#370)

Changes Sphinx config handling for options, which can take a function.

* Introduces Sphinx-Needs Config opbject

Fixes #343

Makefile

@@ -10,7 +10,11 @@ lint:
 
 .PHONY: test
 test:
-	poetry run nosetests -w tests
+	poetry run nosetests -v -w tests
+
+.PHONY: test
+test-short:
+	poetry run nosetests -v -w tests -I .*official.*
 
 .PHONY: test-matrix
 test-matrix:

docs/api.rst

@@ -13,6 +13,7 @@ For some implementation ideas, take a look into the Sphinx extension
 `Sphinx-Test-Reports <https://sphinx-test-reports.readthedocs.io/en/latest/>`_ and its
 `source code <https://github.com/useblocks/sphinx-test-reports/blob/master/sphinxcontrib/test_reports/test_reports.py#L51>`_.
 
+.. _api_configuration:
 
 Configuration
 -------------

docs/changelog.rst

@@ -10,16 +10,6 @@ License
 -----
 :Released: under development
 
-* Bugfix: :ref:`needtable` need parts 'id' column is not linked
-  (`#336 <https://github.com/useblocks/sphinxcontrib-needs/issues/336>`_)
-* Bugfix: :ref:`needtable` need parts 'incoming' column is empty
-  (`#336 <https://github.com/useblocks/sphinxcontrib-needs/issues/336>`_)
-* Bugfix: :ref:`needs_warnings` not written to error log.
-  (`#344 <https://github.com/useblocks/sphinxcontrib-needs/issues/344>`_)
-* Improvement: Providing :ref:`needs_warnings_always_warn` to raise sphinx-warnings for each not passed :ref:`needs_warnings`.
-  (`#344 <https://github.com/useblocks/sphinxcontrib-needs/issues/344>`_)
-* Bugfix: :ref:`needimport` relative path not consistent to Sphinx default directives.
-  (`#351 <https://github.com/useblocks/sphinxcontrib-needs/issues/351>`_)
 * Improvement: New config option :ref:`needs_builder_filter` to define a filter for the needs builder.
   (`#342 <https://github.com/useblocks/sphinxcontrib-needs/issues/342>`_)
 * Improvement: Added option ``json_path`` for :ref:`needs_external_needs` to support external needs from local ``needs.json`` files.
@@ -33,6 +23,18 @@ License
   (`#348 <https://github.com/useblocks/sphinxcontrib-needs/issues/348>`_)
 * Improvement: New config option :ref:`needs_filter_data` to allow to use custom data inside a :ref:`filter_string`
   (`#317 <https://github.com/useblocks/sphinxcontrib-needs/issues/317>`_)
+* Improvement: API to register warnings
+  (`#343 <https://github.com/useblocks/sphinxcontrib-needs/issues/343>`_)
+* Bugfix: :ref:`needtable` need parts 'id' column is not linked
+  (`#336 <https://github.com/useblocks/sphinxcontrib-needs/issues/336>`_)
+* Bugfix: :ref:`needtable` need parts 'incoming' column is empty
+  (`#336 <https://github.com/useblocks/sphinxcontrib-needs/issues/336>`_)
+* Bugfix: :ref:`needs_warnings` not written to error log.
+  (`#344 <https://github.com/useblocks/sphinxcontrib-needs/issues/344>`_)
+* Improvement: Providing :ref:`needs_warnings_always_warn` to raise sphinx-warnings for each not passed :ref:`needs_warnings`.
+  (`#344 <https://github.com/useblocks/sphinxcontrib-needs/issues/344>`_)
+* Bugfix: :ref:`needimport` relative path not consistent to Sphinx default directives.
+  (`#351 <https://github.com/useblocks/sphinxcontrib-needs/issues/351>`_)
 
 0.7.1
 -----

docs/configuration.rst

@@ -3,7 +3,7 @@
 Configuration
 =============
 
-All configurations take place in your project's conf.py file.
+All configurations take place in your project's ``conf.py`` file.
 
 
 .. contents::
@@ -17,6 +17,42 @@ Add **sphinxcontrib.needs** to your extensions::
 
    extensions = ["sphinxcontrib.needs",]
 
+.. _inc_build:
+
+Incremental build support
+-------------------------
+Sphinx does not use its incremental build feature, if functions are assigned directly to Sphinx options.
+To avoid this, please use the :ref:`Sphinx-Needs API <api_configuration>` to register functions directly.
+
+This would allow Sphinx to perform incremental builds, which are much faster as full builds.
+
+**Example configuration**
+
+.. code-block:: python
+
+   # conf.py
+
+   # Defining one or more functions in conf.py is fine
+   def my_custom_warning(need, log):
+       # some checks
+       return False
+
+   def my_dynamic_function(app, need, needs):
+       return "some text"
+
+   # This assignment will deactivate incremental build support inside Sphinx
+   needs_warnings = {
+      'my_warning': my_custom_warning
+   }
+
+   # Better, register the function via Sphinx-Needs API
+   from sphinxcontrib.needs.api.configuration import add_warning, add_dynamic_function
+   def setup(app):
+      add_warning(app, 'my_warning', my_custom_warning)
+      add_dynamic_function(app, my_dynamic_function)
+
+
+
 Options
 -------
 
@@ -98,6 +134,14 @@ It can be specified as a dict inside ``conf.py`` as follows::
     "impacts": directives.unchanged
    }
 
+With version *0.7.2* ``needs_extra_options`` can also be a list and using ``directives.unchanged`` is not needed
+anymore. This does not break the Sphinx incremental build feature. Please read :ref:`inc_build` for details.
+
+.. code-block:: python
+
+   needs_extra_options = ['introduced', 'updated', 'impacts']
+
+
 And used like:
 
 .. code-block:: rst
@@ -423,10 +467,10 @@ Configuration example::
    }
 
 
-The defined ``needs_filter_data`` must be a dictionary. Its values can be a string variable or a custom defined 
+The defined ``needs_filter_data`` must be a dictionary. Its values can be a string variable or a custom defined
 function. The function get execued during config loading and must return a string.
 
-The value of ``needs_filter_data`` will be available as data inside :ref:`filter_string` and can be very poweful together with 
+The value of ``needs_filter_data`` will be available as data inside :ref:`filter_string` and can be very poweful together with
 internal needs info to filter needs.
 
 The defined extra filter data can be used like this::
@@ -967,6 +1011,10 @@ Inside your ``conf.py`` file ue it like this:
 
 See :ref:`dynamic_functions` for ore information.
 
+.. warning::
+
+   Assigning a function to a Sphinx option will deactivate the incremental build feature of Sphinx.
+   Please use the :ref:`Sphinx-Needs API <api_configuration>` and read :ref:`inc_build` for details.
 
 .. _needs_part_prefix:
 
@@ -1009,7 +1057,7 @@ This will handle **all warnings** as exceptions.
            return True
        return False
 
-    
+
    needs_warnings = {
      # req need must not have an empty status field
      'req_with_no_status': "type == 'req' and not status",
@@ -1025,9 +1073,14 @@ This will handle **all warnings** as exceptions.
 The **dictionary key** is used as identifier and gets printed in log outputs.
 The **value** must be a valid filter string or a custom defined filter code function and defines a *not allowed behavior*.
 
-So use the filter string or filter code function to define how needs are not allowed to be configured/used. The defined filter 
+So use the filter string or filter code function to define how needs are not allowed to be configured/used. The defined filter
 code function must return ``True`` or ``False``.
 
+.. warning::
+
+   Assigning a function to a Sphinx option will deactivate the incremental build feature of Sphinx.
+   Please use the :ref:`Sphinx-Needs API <api_configuration>` and read :ref:`inc_build` for details.
+
 Example output:
 
 .. code-block:: text
@@ -1042,7 +1095,7 @@ Example output:
     invalid_status: failed
         failed needs: 11 (STYLE_005, EX_ROW_1, EX_ROW_3, copy_2, clv_1, clv_2, clv_3, clv_4, clv_5, T_C3893, R_AD4A0)
         used filter: status not in ["open", "in progress", "closed", "done"] and status is not None
-    
+
     type_match: failed
         failed needs: 1 (TC_001)
         used filter: <function my_custom_warning_check at 0x7faf3fbcd1f0>

performance/performance_test.py

@@ -19,7 +19,9 @@ def cli():
     pass
 
 
-def start(needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False, browser=False, debug=False):
+def start(
+    needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False, browser=False, debug=False, basic=False
+):
     """
     Test run implementation
     """
@@ -29,6 +31,24 @@ def start(needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False,
 
     shutil.copytree(source_path, source_tmp_path, dirs_exist_ok=True)
 
+    # Render conf.py
+    source_tmp_path_conf = os.path.join(source_tmp_path, "conf.template")
+    source_tmp_path_conf_final = os.path.join(source_tmp_path, "conf.py")
+    template = Template(open(source_tmp_path_conf).read())
+    rendered = template.render(
+        pages=pages,
+        needs=needs,
+        needtables=needtables,
+        dummies=dummies,
+        parallel=parallel,
+        keep=keep,
+        browser=browser,
+        debug=debug,
+        basic=basic,
+    )
+    with open(source_tmp_path_conf_final, "w") as file:
+        file.write(rendered)
+
     # Render index files
     source_tmp_path_index = os.path.join(source_tmp_path, "index.template")
     source_tmp_path_index_final = os.path.join(source_tmp_path, "index.rst")
@@ -44,6 +64,7 @@ def start(needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False,
         keep=keep,
         browser=browser,
         debug=debug,
+        basic=basic,
     )
     with open(source_tmp_path_index_final, "w") as file:
         file.write(rendered)
@@ -65,6 +86,7 @@ def start(needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False,
             keep=keep,
             browser=browser,
             debug=debug,
+            basic=basic,
         )
         with open(source_tmp_path_page_final, "w") as file:
             file.write(rendered)
@@ -156,6 +178,7 @@ def start(needs=1000, needtables=0, dummies=0, pages=1, parallel=1, keep=False,
 @click.option("--browser", is_flag=True, help="Opens the project in your browser")
 @click.option("--snakeviz", is_flag=True, help="Opens snakeviz view for measured profiles in browser")
 @click.option("--debug", is_flag=True, help="Prints more information, incl. sphinx build output")
+@click.option("--basic", is_flag=True, help="Use only default config of Sphinx-Needs (e.g. no extra options)")
 def series(
     profile,
     needs,
@@ -167,6 +190,7 @@ def series(
     browser=False,
     snakeviz=False,
     debug=False,
+    basic=False,
 ):
     """
     Generate and start a series of tests.
@@ -190,6 +214,7 @@ def series(
                         "keep": keep,
                         "browser": browser,
                         "debug": debug,
+                        "basic": basic,
                     },
                 )
 

performance/project/conf.py → performance/project/conf.template

@@ -34,26 +34,13 @@
 # ones.
 
 
-# def rstjinja(app, docname, source):
-#     """
-#     Render our pages as a jinja template for fancy templating goodness.
-#     """
-#     # Make sure we're outputting HTML
-#     if app.builder.format != "html":
-#         return
-#     src = source[0]
-#     rendered = app.builder.templates.render_string(src, app.config.html_context)
-#     source[0] = rendered
-#
-#
-# def setup(app):
-#     app.connect("source-read", rstjinja)
-
-
 extensions = ["sphinxcontrib.needs"]
 
 needs_profiling = ["needtable"]
+
+{% if not basic %}
 needs_types = [
+    dict(directive="req", title="Requirement", prefix="R_", color="#BFD8D2", style="node"),
     dict(directive="story", title="User Story", prefix="US_", color="#BFD8D2", style="node"),
     dict(directive="spec", title="Specification", prefix="SP_", color="#FEDCD2", style="node"),
     dict(directive="impl", title="Implementation", prefix="IM_", color="#DF744A", style="node"),
@@ -65,6 +52,7 @@
 needs_extra_options = {
     "number": directives.unchanged,
 }
+{% endif %}
 
 plantuml = "java -jar %s" % os.path.join(os.path.dirname(__file__), "..", "..", "docs", "utils", "plantuml.jar")
 plantuml_output_format = "svg"

performance/project/page.template

@@ -22,20 +22,25 @@ Needs
 Amount of needs: **{{needs}}**
 
 {% for n in range(needs) %}
-.. story:: Test Need Page {{ page }} {{n}}
-   :id: S_{{page}}_{{n}}
-   :number: {{n}}
-   :links: S_{{page}}_{{needs-n-1}}
+.. req:: Test Need Page {{ page }} {{n}}
+   :id: R_{{page}}_{{n}}
+{% if not basic %}   :number: {{n}}{% endif %}
+   :links: R_{{page}}_{{needs-n-1}}
 {% endfor %}
 
 Needtable
 ~~~~~~~~~
 Amount of needtables: **{{needtables}}**
 
+{%  if basic %}
+.. needtable::
+   :show_filters:
+   :columns: id, title, number, links
+{% else %}
 {% for n in range(needtables) %}
 .. needtable::
    :show_filters:
    :filter: int(number)**3 > 0 or len(links) > 0
    :columns: id, title, number, links
 {% endfor %}
-
+{% endif %}