Add support for meta description (#72)

.github/workflows/workflow.yml

@@ -4,14 +4,17 @@ on:
     branches:
       - main
   push:
-    branches:
-      - main
   create:
     tags:
       - '*'
 jobs:
   check:
     runs-on: ubuntu-latest
+
+    # We want to run on external PRs, but not on our own internal PRs as they'll be run
+    # by the push to the branch.
+    if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
+
     steps:
     - uses: actions/checkout@v3
     - uses: actions/setup-python@v4

README.md

@@ -1,12 +1,15 @@
 # sphinxext-opengraph
-![Build](https://github.com/wpilibsuite/sphinxext-opengraph/workflows/Test%20and%20Deploy/badge.svg)
-[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
-Sphinx extension to generate OpenGraph metadata (https://ogp.me/)
+[![Build](https://github.com/wpilibsuite/sphinxext-opengraph/workflows/Test%20and%20Deploy/badge.svg)](https://github.com/wpilibsuite/sphinxext-opengraph/actions)
+[![Code style: Black](https://img.shields.io/badge/code%20style-Black-000000.svg)](https://github.com/psf/black)
+
+Sphinx extension to generate [Open Graph metadata](https://ogp.me/).
 
 ## Installation
 
-`python -m pip install sphinxext-opengraph`
+```sh
+python -m pip install sphinxext-opengraph
+```
 
 ## Usage
 Just add `sphinxext.opengraph` to your extensions list in your `conf.py`
@@ -17,9 +20,9 @@ extensions = [
 ]
 ```
 ## Options
-These values are placed in the conf.py of your sphinx project.
+These values are placed in the `conf.py` of your Sphinx project.
 
-Users hosting documentation on Read The Docs *do not* need to set any of the following unless custom configuration is wanted. The extension will automatically retrieve your site url.
+Users hosting documentation on Read The Docs *do not* need to set any of the following unless custom configuration is wanted. The extension will automatically retrieve your site URL.
 
 * `ogp_site_url`
     * This config option is very important, set it to the URL the site is being hosted on. 
@@ -32,12 +35,14 @@ Users hosting documentation on Read The Docs *do not* need to set any of the fol
 * `ogp_image_alt`
     * This is not required. Alt text for image. Defaults to using `ogp_site_name` or the document's title as alt text, if available. Set to `False` if you want to turn off alt text completely.
 * `ogp_use_first_image`
-    * This is not required. Set to True to use each page's first image, if available. If set to True but no image is found, Sphinx will use `ogp_image` instead.
+    * This is not required. Set to `True` to use each page's first image, if available. If set to `True` but no image is found, Sphinx will use `ogp_image` instead.
 * `ogp_type`
-    * This sets the ogp type attribute, for more information on the types available please take a look at https://ogp.me/#types. By default it is set to `website`, which should be fine for most use cases.
+    * This sets the ogp type attribute, for more information on the types available please take a look at [https://ogp.me/#types](https://ogp.me/#types). By default it is set to `website`, which should be fine for most use cases.
 * `ogp_custom_meta_tags`
     * This is not required. List of custom html snippets to insert.
-
+* `ogp_enable_meta_description`
+    * This is not required. When `True`, generates `<meta name="description" content="...">` from the page.
+
 ## Example Config
 
 ### Simple Config
@@ -59,20 +64,23 @@ ogp_custom_meta_tags = [
     '<meta property="og:ignore_canonical" content="true" />',
 ]
 
+ogp_enable_meta_description = True
 ```
 
 ## Per Page Overrides
-[Field lists](https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html) are used to allow you to override certain settings on each page and set unsupported arbitrary OpenGraph tags.
+[Field lists](https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html) are used to allow you to override certain settings on each page and set unsupported arbitrary Open Graph tags.
 
 Make sure you place the fields at the very start of the document such that Sphinx will pick them up and also won't build them into the html.
 
 ### Overrides
-These are some overrides that can be used, you can actually override any tag and field lists will always take priority.
+These are some overrides that can be used on individual pages, you can actually override any tag and field lists will always take priority.
 
 * `:og_description_length:`
-  * Configure the amount of characters to grab for the description of the page. If the value isn't a number it will fall back to `ogp_description_length`. Note the slightly different syntax because this isn't directly an OpenGraph tag.
+  * Configure the amount of characters to grab for the description of the page. If the value isn't a number it will fall back to `ogp_description_length`. Note the slightly different syntax because this isn't directly an Open Graph tag.
 * `:og:description:`
   * Lets you override the description of the page.
+* `:description:` or `.. meta::\n :description:`
+  * Sets the `<meta name="description" content="...">` description.
 * `:og:title:`
   * Lets you override the title of the page.
 * `:og:type:`
@@ -95,7 +103,7 @@ Page contents
 ```
 
 ### Arbitrary Tags[^1]
-Additionally, you can use field lists to add any arbitrary OpenGraph tag not supported by the extension. The syntax for arbitrary tags is the same with `:og:tag: content`. For Example:
+Additionally, you can use field lists to add any arbitrary Open Graph tag not supported by the extension. The syntax for arbitrary tags is the same with `:og:tag: content`. For example:
 
 ```rst
 :og:video: http://example.org/video.mp4

docs/source/conf.py

@@ -44,7 +44,6 @@
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = []
 
-
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for

sphinxext/opengraph/__init__.py

@@ -6,6 +6,7 @@
 from sphinx.application import Sphinx
 
 from .descriptionparser import get_description
+from .metaparser import get_meta_description
 from .titleparser import get_title
 
 import os
@@ -28,10 +29,10 @@
 }
 
 
-def make_tag(property: str, content: str) -> str:
+def make_tag(property: str, content: str, type_: str = "property") -> str:
     # Parse quotation, so they won't break html tags if smart quotes are disabled
     content = content.replace('"', """)
-    return f'<meta property="{property}" content="{content}" />'
+    return f'<meta {type_}="{property}" content="{content}" />'
 
 
 def get_tags(
@@ -45,6 +46,7 @@ def get_tags(
     if fields is None:
         fields = {}
     tags = {}
+    meta_tags = {}  # For non-og meta tags
 
     # Set length of description
     try:
@@ -105,6 +107,11 @@ def get_tags(
     if description:
         tags["og:description"] = description
 
+        if config["ogp_enable_meta_description"] and not get_meta_description(
+            context["metatags"]
+        ):
+            meta_tags["description"] = description
+
     # image tag
     # Get basic values from config
     if "og:image" in fields:
@@ -160,7 +167,9 @@ def get_tags(
 
     return (
         "\n".join(
-            [make_tag(p, c) for p, c in tags.items()] + config["ogp_custom_meta_tags"]
+            [make_tag(p, c) for p, c in tags.items()]
+            + [make_tag(p, c, "name") for p, c in meta_tags.items()]
+            + config["ogp_custom_meta_tags"]
         )
         + "\n"
     )
@@ -186,6 +195,7 @@ def setup(app: Sphinx) -> Dict[str, Any]:
     app.add_config_value("ogp_type", "website", "html")
     app.add_config_value("ogp_site_name", None, "html")
     app.add_config_value("ogp_custom_meta_tags", [], "html")
+    app.add_config_value("ogp_enable_meta_description", True, "html")
 
     app.connect("html-page-context", html_page_context)
 

sphinxext/opengraph/metaparser.py

@@ -0,0 +1,29 @@
+from html.parser import HTMLParser
+
+
+class HTMLTextParser(HTMLParser):
+    """
+    Parse HTML into text
+    """
+
+    def __init__(self):
+        super().__init__()
+        self.meta_description = None
+
+    def handle_starttag(self, tag, attrs) -> None:
+        # For example:
+        # attrs = [("content", "My manual description"), ("name", "description")]
+        if ("name", "description") in attrs:
+            self.meta_description = True
+            for name, value in attrs:
+                if name == "content":
+                    self.meta_description = value
+                    break
+
+
+def get_meta_description(meta_tags: str) -> bool:
+    htp = HTMLTextParser()
+    htp.feed(meta_tags)
+    htp.close()
+
+    return htp.meta_description

tests/roots/test-meta-name-description-manual-description/conf.py

@@ -0,0 +1,10 @@
+extensions = ["sphinxext.opengraph"]
+
+master_doc = "index"
+exclude_patterns = ["_build"]
+
+html_theme = "basic"
+
+ogp_site_url = "http://example.org/en/latest/"
+
+ogp_enable_meta_description = True

tests/roots/test-meta-name-description-manual-description/index.rst

@@ -0,0 +1,4 @@
+.. meta::
+   :description: My manual description
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse at lorem ornare, fringilla massa nec, venenatis mi. Donec erat sapien, tincidunt nec rhoncus nec, scelerisque id diam. Orci varius natoque penatibus et magnis dis parturient mauris.

tests/roots/test-meta-name-description-manual-og-description/conf.py

@@ -0,0 +1,10 @@
+extensions = ["sphinxext.opengraph"]
+
+master_doc = "index"
+exclude_patterns = ["_build"]
+
+html_theme = "basic"
+
+ogp_site_url = "http://example.org/en/latest/"
+
+ogp_enable_meta_description = True

tests/roots/test-meta-name-description-manual-og-description/index.rst

@@ -0,0 +1,3 @@
+:og:description: My manual og:description
+
+Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse at lorem ornare, fringilla massa nec, venenatis mi. Donec erat sapien, tincidunt nec rhoncus nec, scelerisque id diam. Orci varius natoque penatibus et magnis dis parturient mauris.

tests/roots/test-meta-name-description/conf.py

@@ -0,0 +1,10 @@
+extensions = ["sphinxext.opengraph"]
+
+master_doc = "index"
+exclude_patterns = ["_build"]
+
+html_theme = "basic"
+
+ogp_site_url = "http://example.org/en/latest/"
+
+enable_meta_description = True

tests/roots/test-meta-name-description/index.rst

@@ -0,0 +1 @@
+Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse at lorem ornare, fringilla massa nec, venenatis mi. Donec erat sapien, tincidunt nec rhoncus nec, scelerisque id diam. Orci varius natoque penatibus et magnis dis parturient mauris.

tests/test_options.py

@@ -1,7 +1,6 @@
 import pytest
 from sphinx.application import Sphinx
 import conftest
-import os
 
 
 def get_tag(tags, tag_type):
@@ -13,6 +12,12 @@ def get_tag_content(tags, tag_type):
     return get_tag(tags, tag_type).get("content", "")
 
 
+def get_meta_description(tags):
+    return [tag for tag in tags if tag.get("name") == "description"][0].get(
+        "content", ""
+    )
+
+
 @pytest.mark.sphinx("html", testroot="simple")
 def test_simple(og_meta_tags):
     description = get_tag_content(og_meta_tags, "description")
@@ -26,6 +31,32 @@ def test_simple(og_meta_tags):
     )
 
 
+@pytest.mark.sphinx("html", testroot="meta-name-description")
+def test_meta_name_description(meta_tags):
+    og_description = get_tag_content(meta_tags, "description")
+    description = get_meta_description(meta_tags)
+
+    assert description == og_description
+
+
+@pytest.mark.sphinx("html", testroot="meta-name-description-manual-description")
+def test_meta_name_description(meta_tags):
+    og_description = get_tag_content(meta_tags, "description")
+    description = get_meta_description(meta_tags)
+
+    assert description != og_description
+    assert description == "My manual description"
+
+
+@pytest.mark.sphinx("html", testroot="meta-name-description-manual-og-description")
+def test_meta_name_description(meta_tags):
+    og_description = get_tag_content(meta_tags, "description")
+    description = get_meta_description(meta_tags)
+
+    assert og_description != description
+    assert og_description == "My manual og:description"
+
+
 @pytest.mark.sphinx("html", testroot="simple")
 def test_site_url(og_meta_tags):
     # Uses the same directory as simple, because it already contains url for a minimal config