From b62ba3e3b7a75641319057bc975e85f5e7c7e1c8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 6 Aug 2020 15:31:19 +0200 Subject: [PATCH 1/5] Fix "lanuages" typo in docs --- docs/extensions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/extensions.rst b/docs/extensions.rst index 64cb632..a60f31c 100644 --- a/docs/extensions.rst +++ b/docs/extensions.rst @@ -1,7 +1,7 @@ Extensions ---------- -Code Annotations uses `Stevedore`_ to allow new lanuages to be statically searched in an easily extensible fashion. All +Code Annotations uses `Stevedore`_ to allow new languages to be statically searched in an easily extensible fashion. All language searches, even the ones that come by default, are implemented as extensions. A language extension is responsible for finding all comments in files of the given type. Note that extensions are only used in the Static Search and not in Django Search, as Django models are obviously all written in Python. From d11aae9ee74727c8935cee9d49d1c4b9d432e8fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 6 Aug 2020 15:38:59 +0200 Subject: [PATCH 2/5] Remove outdated AUTHORS requirement in github PR template --- .github/PULL_REQUEST_TEMPLATE.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 048fb58..ecd45b4 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -2,11 +2,11 @@ **JIRA:** Link to JIRA ticket -**Dependencies:** dependencies on other outstanding PRs, issues, etc. +**Dependencies:** dependencies on other outstanding PRs, issues, etc. **Merge deadline:** List merge deadline (if any) -**Installation instructions:** List any non-trivial installation +**Installation instructions:** List any non-trivial installation instructions. **Testing instructions:** @@ -16,8 +16,8 @@ instructions. 3. If C happened instead - check failed. **Reviewers:** -- [ ] tag reviewer -- [ ] tag reviewer +- [ ] tag reviewer +- [ ] tag reviewer **Merge checklist:** - [ ] All reviewers approved @@ -26,14 +26,13 @@ instructions. - [ ] Changelog record added - [ ] Documentation updated (not only docstrings) - [ ] Commits are squashed -- [ ] PR author is listed in AUTHORS **Post merge:** - [ ] Create a tag -- [ ] Check new version is pushed to PyPi after tag-triggered build is +- [ ] Check new version is pushed to PyPi after tag-triggered build is finished. - [ ] Delete working branch (if not needed anymore) -**Author concerns:** List any concerns about this PR - inelegant -solutions, hacks, quick-and-dirty implementations, concerns about +**Author concerns:** List any concerns about this PR - inelegant +solutions, hacks, quick-and-dirty implementations, concerns about migrations, etc. From 4b95c8d56a34d2b22d3a0ff2e74fb9897b971d9d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 6 Aug 2020 15:00:31 +0200 Subject: [PATCH 3/5] [BD-21] Include "config_and_tools" folder in pip-installable package When pip-installing code-annotations, the config_and_tools folder should also be installed, along with all configuration files. To achieve this, we do the following: - Move the "config_and_tools" folder to the code_annotations folder. That way we can run in python: from code_annotations.config_and_tools.sphinx.extensions import featuretoggles - Add all ".yaml" and ".yml" files to the source distribution by mean of the MANIFEST.in --- CHANGELOG.rst | 1 + MANIFEST.in | 2 +- .../config_and_tools}/README.rst | 0 3 files changed, 2 insertions(+), 1 deletion(-) rename {config_and_tools => code_annotations/config_and_tools}/README.rst (100%) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index f8d1e31..43933f5 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -14,6 +14,7 @@ Change Log Unreleased ~~~~~~~~~~ +* Include ``config_and_tools`` folder in pip-installable package * Add ADR 0001-config-and-tools.rst for adding a placing in this repository for shared annotation configs and supporting tools. [0.4.0] - 2020-07-22 diff --git a/MANIFEST.in b/MANIFEST.in index 935a604..08f31b1 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -4,4 +4,4 @@ include CONTRIBUTING.rst include LICENSE.txt include README.rst include requirements/base.in -recursive-include code_annotations *.html *.png *.gif *js *.css *jpg *jpeg *svg *py +recursive-include code_annotations *.html *.png *.gif *js *.css *jpg *jpeg *svg *py *.yaml *.yml diff --git a/config_and_tools/README.rst b/code_annotations/config_and_tools/README.rst similarity index 100% rename from config_and_tools/README.rst rename to code_annotations/config_and_tools/README.rst From 90dbf1c662a99f6a4c2cd2727ef130068b8e97d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 6 Aug 2020 15:10:54 +0200 Subject: [PATCH 4/5] [BD-21] Add featuretoggle configuration and sphinx extension We introduce a new Sphinx extension for generating human-readable documentation of feature toggle annotations. This extension introduces a `.. featuretoggles::` directive that will list feature toggles for the configured code base. Annotations are collected on-the-fly using the feature toggle annotation configuration file. --- CHANGELOG.rst | 5 +- code_annotations/__init__.py | 2 +- .../config/feature_toggle_annotations.yaml | 28 ++++ .../sphinx/extensions/featuretoggles.py | 151 ++++++++++++++++++ docs/index.rst | 1 + docs/sphinx_extensions.rst | 22 +++ 6 files changed, 206 insertions(+), 3 deletions(-) create mode 100644 code_annotations/config_and_tools/config/feature_toggle_annotations.yaml create mode 100644 code_annotations/config_and_tools/sphinx/extensions/featuretoggles.py create mode 100644 docs/sphinx_extensions.rst diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 43933f5..10ad0b4 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -11,9 +11,10 @@ Change Log .. There should always be an "Unreleased" section for changes pending release. -Unreleased -~~~~~~~~~~ +[0.5.0] - 2020-08-06 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +* Add ``featuretoggles`` Sphinx extension * Include ``config_and_tools`` folder in pip-installable package * Add ADR 0001-config-and-tools.rst for adding a placing in this repository for shared annotation configs and supporting tools. diff --git a/code_annotations/__init__.py b/code_annotations/__init__.py index d45c92b..e0747a9 100644 --- a/code_annotations/__init__.py +++ b/code_annotations/__init__.py @@ -2,4 +2,4 @@ Extensible tools for parsing annotations in codebases. """ -__version__ = '0.4.0' +__version__ = '0.5.0' diff --git a/code_annotations/config_and_tools/config/feature_toggle_annotations.yaml b/code_annotations/config_and_tools/config/feature_toggle_annotations.yaml new file mode 100644 index 0000000..4d2aeeb --- /dev/null +++ b/code_annotations/config_and_tools/config/feature_toggle_annotations.yaml @@ -0,0 +1,28 @@ +# This code-annotations configuration file supports OEP-17, Feature Toggles. + +source_path: ./ +report_path: reports +safelist_path: .annotation_safe_list.yml +coverage_target: 50.0 +annotations: + documented_elsewhere: + - ".. documented_elsewhere:": + - ".. documented_elsewhere_name:": + feature_toggle: + - ".. toggle_name:": + - ".. toggle_implementation:": + choices: [ExperimentWaffleFlag, WaffleFlag, WaffleSample, WaffleSwitch, CourseWaffleFlag, ConfigurationModel, DjangoSetting] + - ".. toggle_default:": + - ".. toggle_description:": + - ".. toggle_category:": + - ".. toggle_use_cases:": + choices: [incremental_release, launch_date, monitored_rollout, graceful_degradation, beta_testing, vip, opt_out, opt_in, open_edx] + - ".. toggle_creation_date:": + - ".. toggle_expiration_date:": + - ".. toggle_warnings:": + - ".. toggle_tickets:": + - ".. toggle_status:": +extensions: + python: + - py +rst_template: doc.rst.j2 diff --git a/code_annotations/config_and_tools/sphinx/extensions/featuretoggles.py b/code_annotations/config_and_tools/sphinx/extensions/featuretoggles.py new file mode 100644 index 0000000..3ad152e --- /dev/null +++ b/code_annotations/config_and_tools/sphinx/extensions/featuretoggles.py @@ -0,0 +1,151 @@ +""" +Sphinx extension for viewing feature toggle annotations. +""" +import os + +import pkg_resources + +from code_annotations.base import AnnotationConfig +from code_annotations.find_static import StaticSearch +from docutils import nodes +from sphinx.util.docutils import SphinxDirective + + +def find_feature_toggles(source_path): + """ + Find the feature toggles as defined in the configuration file. + + Return: + toggles (dict): feature toggles indexed by name. + """ + config_path = pkg_resources.resource_filename( + "code_annotations", + os.path.join("config_and_tools", "config", "feature_toggle_annotations.yaml"), + ) + config = AnnotationConfig( + config_path, verbosity=0, source_path_override=source_path + ) + results = StaticSearch(config).search() + + toggles = {} + current_entry = {} + for filename, entries in results.items(): + for entry in entries: + key = entry["annotation_token"] + value = entry["annotation_data"] + if key == ".. toggle_name:": + toggle_name = value + toggles[toggle_name] = { + "filename": filename, + "line_number": entry["line_number"], + } + current_entry = toggles[toggle_name] + else: + current_entry[key] = value + + return toggles + + +class FeatureToggles(SphinxDirective): + """ + Sphinx directive to list the feature toggles in a single documentation page. + + Use this directive as follows:: + + .. featuretoggles:: + + This directive supports the following configuration parameters: + + - ``featuretoggles_source_path``: absolute path to the repository file tree. E.g: + + featuretoggles_source_path = os.path.join(os.path.dirname(__file__), "..", "..") + + - ``featuretoggles_repo_url``: Github repository where the code is hosted. E.g: + + featuretoggles_repo_url = "https://github.com/edx/myrepo" + + - ``featuretoggles_repo_version``: current version of the git repository. E.g: + + import git + try: + repo = git.Repo(search_parent_directories=True) + featuretoggles_repo_version = repo.head.object.hexsha + except git.InvalidGitRepositoryError: + featuretoggles_repo_version = "master" + """ + + required_arguments = 0 + optional_arguments = 0 + option_spec = {} + + def run(self): + """ + Public interface of the Directive class. + + Return: + nodes (list): nodes to be appended to the resulting document. + """ + toggle_nodes = list(self.iter_nodes()) + return [nodes.section("", *toggle_nodes, ids=["featuretoggles"])] + + def iter_nodes(self): + """ + Iterate on the docutils nodes generated by this directive. + """ + toggles = find_feature_toggles(self.env.config.featuretoggles_source_path) + for toggle_name in sorted(toggles): + toggle = toggles[toggle_name] + yield nodes.title(text=toggle_name) + toggle_default_value = toggle.get(".. toggle_default:", "Not defined") + toggle_default_node = nodes.literal(text=quote_value(toggle_default_value)) + yield nodes.paragraph("", "Default: ", toggle_default_node) + yield nodes.paragraph( + "", + "Source: ", + nodes.reference( + text="{} (line {})".format( + toggle["filename"], toggle["line_number"] + ), + refuri="{}/blob/{}/{}#L{}".format( + self.env.config.featuretoggles_repo_url, + self.env.config.featuretoggles_repo_version, + toggle["filename"], + toggle["line_number"], + ), + ), + ) + yield nodes.paragraph(text=toggle.get(".. toggle_description:", "")) + + +def quote_value(value): + """ + Quote a Python object if it is string-like. + """ + if value in ("True", "False", "None"): + return str(value) + try: + float(value) + return str(value) + except ValueError: + pass + if isinstance(value, str): + return '"{}"'.format(value) + return str(value) + + +def setup(app): + """ + Declare the Sphinx extension. + """ + app.add_config_value( + "featuretoggles_source_path", os.path.abspath(".."), "env", + ) + app.add_config_value("featuretoggles_repo_url", "", "env") + app.add_config_value("featuretoggles_repo_version", "master", "env") + app.add_directive("featuretoggles", FeatureToggles) + + return { + "version": "0.1", + "parallel_read_safe": True, + "parallel_write_safe": True, + } diff --git a/docs/index.rst b/docs/index.rst index 8cfef45..678ee14 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -18,6 +18,7 @@ Contents: static_search django_search configuration + sphinx_extensions extensions testing modules diff --git a/docs/sphinx_extensions.rst b/docs/sphinx_extensions.rst new file mode 100644 index 0000000..35b7bbe --- /dev/null +++ b/docs/sphinx_extensions.rst @@ -0,0 +1,22 @@ +Sphinx extensions +----------------- + +``featuretoggles`` +================== + +This package can be used to document the feature toggles in your code base. To do so, +add the following to your ``conf.py``:: + + extensions = ["code_annotations.config_and_tools.sphinx.extensions.featuretoggles"] + featuretoggles_source_path = os.path.abspath( + os.path.join(os.path.dirname(__file__), "..") + ) + featuretoggles_repo_url = "https://github.com/edx/yourrepo" + try: + featuretoggles_repo_version = git.Repo(search_parent_directories=True).head.object.hexsha + except git.InvalidGitRepositoryError: + pass + +Then, in an ``.rst`` file:: + + .. featuretoggles:: From cb555d6375388d7131ddfdc7545ce4812bf2bbb0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Behmo?= Date: Thu, 13 Aug 2020 00:18:12 +0200 Subject: [PATCH 5/5] Fix minor typo in changelog --- CHANGELOG.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 10ad0b4..7d0dd47 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -16,7 +16,7 @@ Change Log * Add ``featuretoggles`` Sphinx extension * Include ``config_and_tools`` folder in pip-installable package -* Add ADR 0001-config-and-tools.rst for adding a placing in this repository for shared annotation configs and supporting tools. +* Add ADR 0001-config-and-tools.rst for adding a place in this repository for shared annotation configs and supporting tools. [0.4.0] - 2020-07-22 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~