Merge remote-tracking branch 'origin/main' into adam-glossary

Signed-off-by: Liran Funaro <liran.funaro@gmail.com>

# Conflicts:
#	sphinx_markdown_builder/translator.py

.github/workflows/test.yaml

@@ -20,15 +20,8 @@ jobs:
           python -m pip install --upgrade pip
           pip install --upgrade flake8 pylint
           pip install .[dev]
-      - name: Validate coding conventions with black
-        run: |
-          black sphinx_markdown_builder --check --diff
-      - name: Lint with flake8
-        run: |
-          flake8 . --count --select=E,F,W,C --show-source --max-complexity=10 --max-line-length=120 --statistics
-      - name: Lint with pylint
-        run: |
-          pylint sphinx_markdown_builder --disable C0116,C0115
+      - name: Lint
+        run: make lint
 
   test:
     runs-on: ubuntu-latest
@@ -47,11 +40,7 @@ jobs:
           pip install --upgrade pytest pytest-cov coveralls
           pip install .[dev]
       - name: Test
-        run: |
-          make test
-      - name: Coverage test with pytest
-        run: |
-          pytest --cov=sphinx_markdown_builder
+        run: make test
       - name: Coveralls
         if: ${{ !env.ACT }} # skip during local actions testing
         env:

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-We accept contributions of every kind: documentation, code, artwork. Any help is greatly
+We accept contributions of every kind: documentation, code, and artwork. Any help is greatly
 appreciated. This document contains everything needed to get started with your first contribution.
 
 
@@ -14,36 +14,66 @@ request.
 
 It is usually a good idea to discuss major changes with the developers, this will help us
 determine whether the contribution would be a good fit for the project and if it is likely to be
-accepted. There's nothing worse than seeing your hard work being rejected because it falls outside
-of the scope of the project.
-
-Make sure your editor respects the [EditorConfig](http://editorconfig.org) configuration file we
-put at the root of the repository.
+accepted. There's nothing worse than seeing your hard work being rejected because it falls outside the scope of the project.
 
 We follow [GitHub Flow](http://scottchacon.com/2011/08/31/github-flow.html) as our git workflow of
 choice which boils down to:
 
-* The `master` branch is always stable and deployable.
-* To work on something new, branch off `master` and give the new branch a descriptive name (e.g.:
-  `sort-packages-by-name`, `issue-32`, etc).
-* Regularly __rebase__ that branch against `master` and push your work to a branch with the same
+* The `main` branch is always stable and deployable.
+* To work on something new, branch off `main` and give the new branch a descriptive name (e.g.:
+  `sort-packages-by-name`, `issue-32`, etc.).
+* Regularly __rebase__ that branch against `main` and push your work to a branch with the same
   name on the server.
 * When you need feedback, help or think you are ready,
   [submit a pull request](https://help.github.com/articles/using-pull-requests).
-* Once the branch has been merged (or rebased) into `master`, delete it from both your local and
+* Once the branch has been merged (or rebased) into `main`, delete it from both your local and
   remote repository.
 
 We invite you to follow
 [these guidelines](http://who-t.blogspot.de/2009/12/on-commit-messages.html) to write useful
 commit messages.
 
-Additionally, you don't need to add entries to the [CHANGELOG.md](CHANGELOG.md) file, this is our
-responsibility.
+To make sure modifications do not break this package functionality, we employ a CI workflow that tests the following:
+ - Coding conventions (lint)
+ - Expected outputs (tests)
+ - Test code coverage
 
+It is best to verify this before submitting a PR.
 
-## Contributing Tests
+#### Prepare Environment
+To make sure your environment matches the one in the CI servers, please create a new virtual environment. 
+ For example, [venv](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment):
+```shell
+python3 -m venv .venv
+source .venv/bin/activate
+pip install .[dev]
+```
+Note
+: The specific commands may change based on your local system.
 
+#### Run linters
+To make sure your modifications respect the project's coding standards, please run the following:
+```shell
+make lint
+```
+
+#### Run Tests
+To make sure your modification does not change the expected behavior, please run the following.
+```shell
+make test
+```
 Testing is done with input/output examples.
+If it fails, it will show the difference between generated and expected output.
+
+To better view the difference after running `make test`, you can use the following command:
+```bash
+make diff DIFFTOOL=meld
+```
+You can replace 'DIFFTOOL=meld' with any "diff" tool you have on your local machine. The default is `meld`.
+
+
+## Contributing Tests
+
 Since this project is in its early stages, we lack meaningful examples.
 We encourage you to suggest improvements, fixes, and additions to these tests by opening pull requests.
 
@@ -58,24 +88,9 @@ Important
 : For resolving discrepancies, don't worry about passing CI tests.
 Upon accepting your PR, developers will make additional effort to resolve these discrepancies and make the tests pass.
 
-The following command tests that the generated output matches existing results:
-```bash
-make test
-```
-
-If it fails, it will show the difference between generated and expected output.
-
-To better view the difference after running `make test`, you can use the following command:
-```bash
-make diff DIFFTOOL=meld
-```
-Where you can replace 'DIFFTOOL=meld' with any diff tool you have on your local machine.
-The default is `meld`.
-
 
 ## Reading List
 
 * [GitHub Flow](http://scottchacon.com/2011/08/31/github-flow.html)
-* [Keep a Changelog](http://keepachangelog.com/)
 * [On Commit Messages](http://who-t.blogspot.de/2009/12/on-commit-messages.html)
 * [Semantic Versioning](http://semver.org/)

Makefile

@@ -8,7 +8,7 @@ SOURCE_DIR        = $(TESTS_DIR)/source
 BUILD_DIR         = $(TESTS_DIR)/docs-build
 EXPECTED_DIR      = $(TESTS_DIR)/expected
 
-.PHONY: help clean test meld release
+.PHONY: help clean test test-diff diff meld release
 
 # Put it first so that "make" without argument is like "make help".
 help:
@@ -28,23 +28,44 @@ doc-%:
 docs: doc-markdown
 
 
-test:
+test-diff:
+	@echo "Building markdown..."
 	@$(SPHINX_BUILD) -M markdown "$(SOURCE_DIR)" "$(BUILD_DIR)" $(SPHINX_OPTS) $(O) -a -t Partners
-	@# Build overrides and copy just one file for verification
+
+	@echo "Building markdown with configuration overrides..."
 	@$(SPHINX_BUILD) -M markdown "$(SOURCE_DIR)" "$(BUILD_DIR)/overrides" $(SPHINX_OPTS) $(O) -a \
 			-D markdown_http_base="https://localhost" -D markdown_uri_doc_suffix=".html" \
 			-D markdown_docinfo=True -D markdown_anchor_sections=True -D markdown_anchor_signatures=True
 
+	@# Copy just one file for verification
 	@cp "$(BUILD_DIR)/overrides/markdown/auto-summery.md" "$(BUILD_DIR)/markdown/overrides-auto-summery.md"
 	@rm -r $(BUILD_DIR)/markdown/_static $(BUILD_DIR)/markdown/permalink.html
+
+	@echo "Verifies outputs..."
 	@diff --recursive --color=always --side-by-side --text --suppress-common-lines \
 			"$(BUILD_DIR)/markdown" "$(EXPECTED_DIR)"
 
 
+test: test-diff
+	@echo "Unit testing and coverage report..."
+	@pytest --cov=sphinx_markdown_builder
+
+
 diff:
 	$(DIFFTOOL) "$(BUILD_DIR)/markdown" "$(EXPECTED_DIR)" &
 
 
+lint:
+	@echo "Validate coding conventions with black"
+	black sphinx_markdown_builder --check --diff
+	@echo "Lint with flake8"
+	flake8 . --count --select=E,F,W,C --show-source \
+			--max-complexity=10 --max-line-length=120 --statistics \
+			--exclude "venv,.venv,.git"
+	@ echo "Lint with pylint"
+	pylint sphinx_markdown_builder --disable C0116,C0115
+
+
 release:
 	@rm -rf dist/*
 	python3 -m build || exit

README.md

@@ -46,6 +46,11 @@ markdown_uri_doc_suffix = ".html"
 
 Then a reference to `your-doc-name#your-header` will be substituted with `https://your-domain.com/docs/your-doc-name.html#your-header`. 
 
+
+## Contributing
+
+See the [code contribution guidelines](CONTRIBUTING.md) for more information.
+
 ## Credits
 This project forked from [clayrisser/sphinx-markdown-builder], which was developed by [Clay Risser] under the [MIT] license.
 

pyproject.toml

@@ -19,7 +19,7 @@ classifiers = [
     "Topic :: Software Development :: Libraries :: Python Modules"
 ]
 keywords = ["sphinx", "sphinx-extention", "markdown", "docs", "documentation", "builder"]
-dependencies = ["sphinx>=2.2.0", "tabulate", "docutils"]
+dependencies = ["sphinx>=5.1.0", "tabulate", "docutils"]
 requires-python = ">=3.7"
 
 [tool.poetry.plugins] # Optional super table
@@ -29,6 +29,7 @@ requires-python = ">=3.7"
 
 [project.optional-dependencies]
 dev = [
+    "sphinx>=5.3.0", # For development, we need a higher version for the tests' outputs to be identical.
     "bumpver", "black", "isort", "flake8", "pylint", "pip-tools", "pytest", "pytest-cov", "coveralls",
     "sphinx-needs", "sphinxcontrib-plantuml",
 ]

sphinx_markdown_builder/contexts.py

@@ -25,15 +25,29 @@ class UniqueString(str):
 DEFAULT_TARGET = "body"
 CONTENT_START = UniqueString("content start")
 EOL = "\n"
+SPACE = " "
 SPACE_CHARS = re.compile(r"\s+")
+LETTERS = re.compile(r"[a-z0-9]", re.I)
 WRAP_REGEXP = re.compile(r"(\s*)(?=\S)([\s\S]+?)(?<=\S)(\s*)", re.M)
 MULTI_LINE_BREAK = re.compile(r"(?<=\n)\n")
 
 
-def is_content_start(value: str):
+def is_content_start(value: str) -> bool:
     return isinstance(value, UniqueString) and value is CONTENT_START
 
 
+def is_space(value: str) -> bool:
+    return SPACE_CHARS.fullmatch(value) is not None
+
+
+def is_eol(value: str) -> bool:
+    return value == EOL
+
+
+def is_letter(value: str) -> bool:
+    return LETTERS.fullmatch(value) is not None
+
+
 def replace_multi_line_break(value: str):
     return MULTI_LINE_BREAK.sub("<br/>\n", value)
 
@@ -86,27 +100,27 @@ def _iter_reverse_char(self) -> Iterator[str]:
 
     def _count_missing_eol(self) -> int:
         """
-        Add required number of EOL characters.
+        Count the number of EOL characters.
         Avoids adding EOL at the beginning of the content.
         Ignores spaces when traversing the content.
         """
         missing_count = self.ensure_eol_count
         for value in self._iter_reverse_char():
             if is_content_start(value):
                 missing_count = 0
-            if missing_count <= 0 or SPACE_CHARS.fullmatch(value) is None:
+            if missing_count <= 0 or not is_space(value):
                 break
 
             # This can only happen if the node's text had trailing EOL.
             # But docutils nodes are expected to be without.
             # So this validation is to avoid redundant EOLs if this behaviour changes in future releases.
-            if value == EOL:
+            if is_eol(value):
                 missing_count -= 1
 
         return max(0, missing_count)
 
     def ensure_eol(self, count: int = 1):
-        """Ensures an EOL will be added before the next appended value"""
+        """Ensures EOLs will be added before the next appended value"""
         self.ensure_eol_count = max(self.ensure_eol_count, count)
 
     def force_eol(self, count: int = 1):
@@ -166,6 +180,10 @@ def make(self):
         # We need to make sure the emphasis mark is near a non-space char,
         # but we want to preserve the existing spaces.
         prefix_space, text, suffix_space = match.groups()
+
+        # Markdown requires italic/bold/etc... to have a space before it if the edge character is not a letter.
+        if self.prefix in ["*", "_"] and not is_letter(text[0]) and len(prefix_space) == 0:
+            prefix_space = SPACE
         return f"{prefix_space}{self.prefix}{text}{self.suffix}{suffix_space}"
 
 
@@ -321,7 +339,7 @@ def make(self):
 
 
 class MetaContext(NoLineBreakContext):
-    def __init__(self, name: str, params=SubContextParams(1, 1, "head")):
+    def __init__(self, name: str, params=SubContextParams(1, 1, target="head")):
         super().__init__("<br/>", params)
         assert name, "Empty meta name"
         self.name = name
@@ -365,67 +383,3 @@ def create(self, node, element_key) -> _ContextT:
     MetaContext,
     translator=lambda _node, elem: {"name": f"{elem}: "},
 )
-
-
-# pylint: disable=too-many-instance-attributes
-class GlossaryContext(SubContext):
-    def __init__(self, params=SubContextParams()):
-        super().__init__(params)
-        self.terms: List[List[str]] = []
-        self.definitions: List[List[str]] = []
-        self.internal_context = SubContext()
-
-        self.old_visit_term = None
-        self.old_depart_term = None
-        self.old_visit_definition = None
-        self.old_depart_definition = None
-
-        self.is_term = False
-        self.is_definition = False
-
-    @property
-    def active_output(self) -> List[List[str]]:
-        if self.is_term:
-            return self.terms
-        assert self.is_definition
-        return self.definitions
-
-    @property
-    def content(self):
-        if self.is_term or self.is_definition:
-            return self.active_output[-1]
-        return self.internal_context.content
-
-    def enter_term(self):
-        assert not self.is_term
-        self.is_term = True
-        self.terms.append([])
-
-    def exit_term(self):
-        assert self.is_term
-        self.is_term = False
-
-    def enter_definition(self):
-        assert not self.is_definition
-        self.is_definition = True
-        self.definitions.append([])
-
-    def exit_definition(self):
-        assert self.is_definition
-        self.is_definition = False
-
-    def make(self):
-        ctx = SubContext()
-        prefix = self.internal_context.make()
-        if prefix:
-            ctx.add(prefix)
-
-        assert len(self.terms) == len(self.definitions)
-
-        # pylint: disable=consider-using-enumerate
-        for i in range(len(self.terms)):
-            refname = "term-" + self.terms[i][1].replace(" ", "-")
-
-            ctx.add(f'<a id="{refname}"></a>\n\n' + "".join(self.terms[i]) + "".join(self.definitions[i]), prefix_eol=2)
-
-        return ctx.make()