Support need items generation from marked RST in source code

docs/source/components/analyse.rst

@@ -40,7 +40,7 @@ Use simplified comment patterns to define **Sphinx-Needs** items without complex
 Marked RST Blocks
 ~~~~~~~~~~~~~~~~~
 
-Embed complete reStructuredText content within source code comments for rich documentation that can be extracted and processed.
+Embed complete reStructuredText directives which is extracted and parsed as the grammar of **Sphinx-Needs** definition blocks.
 
 Limitations
 -----------
@@ -55,6 +55,8 @@ Extraction Examples
 
 The following examples are configured with :ref:`the analyse configuration <analyse_config>`,
 
+.. _`analyse_need_id_refs`:
+
 Sphinx-Needs ID References
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -118,6 +120,8 @@ Below is an example of a C++ source file containing need ID references and the c
 - ``marker`` - The marker string used for identification
 - ``type`` - Type of extraction ("need-id-refs")
 
+.. _`analyse_rst`:
+
 Marked RST Blocks
 ~~~~~~~~~~~~~~~~~
 
@@ -126,6 +130,7 @@ This example demonstrates how the analyse extracts RST blocks from comments.
 .. tabs::
 
    .. code-tab:: cpp
+      :linenos:
 
        #include <iostream>
 
@@ -190,6 +195,8 @@ The module supports both multi-line and single-line RST blocks:
 - **Multi-line blocks**: Use ``@rst`` and ``@endrst`` on separate lines
 - **Single-line blocks**: Use ``@rst content @endrst`` on the same line
 
+.. _`analyse_oneline`:
+
 One-line Needs
 --------------
 
@@ -199,14 +206,70 @@ For comprehensive information about one-line needs configuration and usage, see
 
 **Basic Example:**
 
-.. code-block:: c
 
-   // @Function Implementation, IMPL_001, impl, [REQ_001, REQ_002]
+.. tabs::
+
+   .. code-tab:: c
+      :linenos:
+
+      // @Function Foo, IMPL_1
+      void foo() {}
+
+      // @Function Bar, IMPL_2
+      void bar() {}
 
-This single comment line creates a complete **Sphinx-Needs** item equivalent to:
+      // @Function Baz\, as I want it, IMPL_3
+      void baz() {}
 
-.. code-block:: rst
+   .. code-tab:: json
 
-   .. impl:: Function Implementation
-       :id: IMPL_001
-       :links: REQ_001, REQ_002
+      [
+         {
+            "filepath": "/home/jui-wen/git_repo/ub/sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
+            "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L1",
+            "source_map": {
+                  "start": { "row": 0, "column": 4 },
+                  "end": { "row": 0, "column": 24 }
+            },
+            "tagged_scope": "void foo() {}",
+            "need": {
+                  "title": "Function Foo",
+                  "id": "IMPL_1",
+                  "type": "impl",
+                  "links": []
+            },
+            "type": "need"
+         },
+         {
+            "filepath": "/home/jui-wen/git_repo/ub/sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
+            "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L4",
+            "source_map": {
+                  "start": { "row": 3, "column": 4 },
+                  "end": { "row": 3, "column": 24 }
+            },
+            "tagged_scope": "void bar() {}",
+            "need": {
+                  "title": "Function Bar",
+                  "id": "IMPL_2",
+                  "type": "impl",
+                  "links": []
+            },
+            "type": "need"
+         },
+         {
+            "filepath": "/home/jui-wen/git_repo/ub/sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
+            "remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L7",
+            "source_map": {
+                  "start": { "row": 6, "column": 4 },
+                  "end": { "row": 6, "column": 39 }
+            },
+            "tagged_scope": "void baz() {}",
+            "need": {
+                  "title": "Function Baz, as I want it",
+                  "id": "IMPL_3",
+                  "type": "impl",
+                  "links": []
+            },
+            "type": "need"
+         }
+      ]

docs/source/components/configuration.rst

@@ -373,14 +373,15 @@ Configures how **Sphinx-CodeLinks** analyse source files to extract markers from
    [codelinks.projects.my_project.analyse.marked_rst]
    start_sequence = "@rst"
    end_sequence = "@endrst"
+   link_options = ["links"]
 
 get_need_id_refs
 ^^^^^^^^^^^^^^^^
 
 Enables the extraction of need IDs from source code comments. When enabled, **SourceAnalyse** will parse comments for specific markers that indicate need IDs, allowing them to be extracted for further usages.
 
 **Type:** ``bool``
-**Default:** ``False``
+**Default:** ``True``
 
 .. code-block:: toml
 
@@ -478,6 +479,8 @@ Configuration for Sphinx-Needs ID reference extraction.
 
 - ``markers`` (``list[str]``) - List of marker strings that identify need ID references
 
+.. _analyse_marked_rst:
+
 analyse.marked_rst
 ^^^^^^^^^^^^^^^^^^
 
@@ -491,8 +494,14 @@ Configuration for marked RST block extraction.
    [codelinks.projects.my_project.analyse.marked_rst]
    start_sequence = "@rst"
    end_sequence = "@endrst"
+   strip_leading_sequences = ["*"]
+   indented_spaces = 3
+   link_options = ["links"]
 
 **Configuration fields:**
 
 - ``start_sequence`` (``str``) - Marker that begins an RST block
 - ``end_sequence`` (``str``) - Marker that ends an RST block
+- ``strip_leading_sequences`` (``list[str]``) - List of leading sequences to strip from each line of the RST text between the markers
+- ``indented_spaces`` (``int``) - Number of leading spaces to consider as indentation in the RST text
+- ``link_options`` (``list[str]``) - List of option names whose values should be treated as Sphinx-Needs link fields

docs/source/components/directive.rst

@@ -3,7 +3,29 @@
 Directive
 =========
 
-.. attention:: ``src-trace`` directive currently only supports :ref:`one-line need definition <oneline>`.
+.. attention:: ``src-trace`` directive do NOT supports :ref:`Sphinx-Needs ID Refs <analyse_need_id_refs>`.
+
+``src-trace`` Directive generates Sphinx-Needs items from source code comments. There are two ways to define need items in source code:
+
+1. **One-line need definition**: Define needs in a single line comment.
+
+   Example in C++:
+
+   .. code-block:: cpp
+
+      // @ title, id_123, implementation, [link1, link2]
+
+2. **RST block need definition**: Define needs in a RST block comment.
+
+   Example in C++:
+
+   .. code-block:: cpp
+
+      /*
+       .. implementation:: title
+         :id: id_123
+         :links: link1, link2
+      */
 
 ``CodeLinks`` provides ``src-trace`` directive and it can be used in the following ways:
 

docs/source/components/rst_parser.rst

@@ -0,0 +1,17 @@
+Simplified reStructuredText Parser
+==================================
+
+The :ref:`analyse <analyse>` module provides a simplified parser for reStructuredText (reST) directives using the ``Lark`` parsing library.
+It is designed to only parse the RST text extracted by :ref:`RST markers <analyse_rst>`, focusing on specific directive types and their associated options and content.
+By doing so, the parser avoids the complexity of a full reST parser while still capturing the essential structure needed for Sphinx-Needs integration from the source code.
+
+The parser does't have the Sphinx-Needs directive validation logic. It only checks the syntax of the RST directives and extracts the directive type, argument, options, and content.
+
+**Limitations**
+
+Since the parser does not implement the full reST specification, it has some limitations:
+
+- Comments in the RST text are not supported.
+- The parser expects consistent indentation for options and content blocks.
+- It only takes an inline directive argument/title (no multi-line arguments/titles).
+- It only takes inline option values (no multi-line option values).

docs/source/development/change_log.rst

@@ -3,6 +3,36 @@
 Changelog
 =========
 
+.. _`unreleased`:
+
+Unreleased
+-----------
+
+New and Improved
+................
+
+- ✨ Added RST directive parser for ``analyse`` module to resolve the marked RST into Sphinx-Needs related data.
+
+  Marked RST blocks can be resolved from the marked RST text in source files using the simplified RST directive parser.
+  The resolved RST blocks will be dumped into the JSON output along with other extracted markers.
+  To make the parser more stable, 3 new configuration options are added to control the parsing behavior:
+
+  - ``leading_sequences``: List of leading character sequences to strip from each line.
+
+    This option allows users to specify a list of leading character sequences (e.g., ``*``, ``-``) that should be stripped
+    from each line of the marked RST block before parsing.
+    This is useful for handling comment styles that include leading characters in multi-line comments.
+
+  - ``indented_spaces``: Number of spaces that indicate an indented block.
+  - ``link_options``: List of option names whose values should be converted to lists when parsed.
+
+- ✨ Enhanced the ``src-trace`` directive to support creating need items from marked RST blocks in addition to one-line need definitions.
+
+  The ``src-trace`` directive can now create Sphinx-Needs need items from both one-line need definitions and marked RST blocks.
+  This provides more flexibility for defining needs in source code comments.
+
+:Released: 02.10.2025
+
 .. _`release:1.1.0`:
 
 1.1.0

docs/source/development/roadmap.rst

@@ -17,7 +17,6 @@ Configuration Files
 Source Code Parsing
 -------------------
 
-- Introduce a configurable option to strip leading characters (e.g., ``*``) from commented RST blocks.
 - Enrich tagged scopes with additional metadata.
 - Extend language support by adding parsers for more comment styles, including but not limited to:
 
@@ -28,5 +27,5 @@ Source Code Parsing
 Defining Needs in Source Code
 -----------------------------
 
-- Introduce flexible ways to define ``Sphinx-Needs`` items in source code, such as using raw RST text and multi-line comments.
+- Introduce flexible ways to define ``Sphinx-Needs`` items in source code, such as custom multi-line need comments.
 - Implement a feature to export needs defined in source code to a ``needs.json`` file, improving CI workflows and portability.

docs/source/index.rst

@@ -62,6 +62,7 @@ Contents
    components/cli
    components/configuration
    components/directive
+   components/rst_parser
    components/oneline
    components/analyse
    components/discover

docs/src_trace.toml

@@ -15,6 +15,7 @@ src_dir = "../tests/data/dcdc" # Relative path from this TOML config to the sour
 [codelinks.projects.dcdc.analyse]
 get_need_id_refs = false
 get_oneline_needs = true
+get_rst = true
 
 [codelinks.projects.dcdc.analyse.oneline_comment_style]
 # Configuration for oneline comment style

docs/ubproject.toml

@@ -4,7 +4,7 @@
 ignore = ["block.title_line"]
 
 [needs]
-id_required = true
+id_required = false
 
 [[needs.types]]
 directive = "feature"

pyproject.toml

@@ -29,6 +29,7 @@ dependencies = [
   "tree-sitter~=0.25.1",
   "tree-sitter-c-sharp>=0.23.1",
   "tree-sitter-yaml>=0.7.1",
+  "lark>=1.3.1",
 ]
 
 [build-system]