Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix linkcheck GitHub 429 errors #5396

Merged
merged 3 commits into from
Jul 22, 2020
Merged

Fix linkcheck GitHub 429 errors #5396

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
835c08f
Reformat docs/conf.py with black
rmol Jul 21, 2020
e453b83
Increase docs-linkcheck worker threads, ignore SD issues URLs
rmol Jul 21, 2020
e5d4604
Ignore another GitHub pattern.
rmol Jul 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
71 changes: 38 additions & 33 deletions docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

# Detect if we're being built by Read the Docs
# https://docs.readthedocs.org/en/latest/faq.html#how-do-i-change-behavior-for-read-the-docs
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
on_rtd = os.environ.get("READTHEDOCS", None) == "True"

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
Expand All @@ -31,35 +31,37 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.todo', ]
extensions = [
"sphinx.ext.todo",
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
templates_path = ["_templates"]

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
source_suffix = ".rst"

# The encoding of source files.
# source_encoding = 'utf-8-sig'

# The master toctree document.
master_doc = 'index'
master_doc = "index"

# General information about the project.
project = u'SecureDrop'
copyright = u'2015-2020, Freedom of the Press Foundation'
author = u'SecureDrop Team and Contributors'
project = u"SecureDrop"
copyright = u"2015-2020, Freedom of the Press Foundation"
author = u"SecureDrop Team and Contributors"

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '1.4.1'
version = "1.4.1"
# The full version, including alpha/beta/rc tags.
release = '1.4.1'
release = "1.4.1"

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand All @@ -76,7 +78,7 @@

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build']
exclude_patterns = ["_build"]

# The reST default role (used for this markup: `text`) to use for all
# documents.
Expand All @@ -94,7 +96,7 @@
# show_authors = False

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
pygments_style = "sphinx"

# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
Expand All @@ -111,21 +113,22 @@
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
if on_rtd:
html_theme = 'default'
html_theme = "default"
else:
try:
# If you want to build the docs locally using the RTD theme,
# you may need to install it: ``pip install sphinx_rtd_theme``.
# https://github.com/snide/sphinx_rtd_theme#via-package
import sphinx_rtd_theme

html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
except ImportError:
# This theme is included with Sphinx and is quite nice (based
# on the Pocoo themes), but since we're using the RTD theme
# for the production docs, it's best to use that to avoid
# issues due to discrepancies between the themes.
html_theme = 'alabaster'
html_theme = "alabaster"

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
Expand All @@ -144,7 +147,7 @@

# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = '../securedrop/static/i/favicon.png'
html_logo = "../securedrop/static/i/favicon.png"

# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
Expand Down Expand Up @@ -217,20 +220,17 @@
# html_search_scorer = 'scorer.js'

# Output file base name for HTML help builder.
htmlhelp_basename = 'SecureDropdoc'
htmlhelp_basename = "SecureDropdoc"

# -- Options for LaTeX output ---------------------------------------------

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper',

# The font size ('10pt', '11pt' or '12pt').
# 'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.
# 'preamble': '',

# Latex figure (float) alignment
# 'figure_align': 'htbp',
}
Expand All @@ -239,8 +239,7 @@
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'SecureDrop.tex', u'SecureDrop Documentation',
author, 'manual'),
(master_doc, "SecureDrop.tex", u"SecureDrop Documentation", author, "manual"),
]

# The name of an image file (relative to this directory) to place at the top of
Expand Down Expand Up @@ -268,10 +267,7 @@

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'securedrop', u'SecureDrop Documentation',
[author], 1)
]
man_pages = [(master_doc, "securedrop", u"SecureDrop Documentation", [author], 1)]

# If true, show URL addresses after external links.
# man_show_urls = False
Expand All @@ -283,9 +279,15 @@
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'SecureDrop', u'SecureDrop Documentation',
author, 'SecureDrop', 'One line description of project.',
'Miscellaneous'),
(
master_doc,
"SecureDrop",
u"SecureDrop Documentation",
author,
"SecureDrop",
"One line description of project.",
"Miscellaneous",
),
]

# Documents to append as an appendix to all manuals.
Expand All @@ -303,11 +305,14 @@
# -- Options for linkcheck --

linkcheck_retries = 3
linkcheck_workers = 32

linkcheck_ignore = [
r'http://127.0.0.1(:\d+)?/?',
r'http://localhost(:\d+)?/?',
'https://forum.securedrop.org/admin/users/list/active',
'https://weblate.securedrop.org/projects/securedrop/securedrop/#repository',
'https://github.com/freedomofpress/securedrop-debian-packages-lfs',
r"http://127.0.0.1(:\d+)?/?",
r"http://localhost(:\d+)?/?",
"https://forum.securedrop.org/admin/users/list/active",
"https://weblate.securedrop.org/projects/securedrop/securedrop/#repository",
"https://github.com/freedomofpress/securedrop-debian-packages-lfs",
r"https://github.com/freedomofpress/securedrop/issues/.*",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No 429s for issues tickets, but I did see one for a path on the develop branch:

Warning, treated as error:
/home/user/securedrop/docs/development/tips_and_tricks.rst.rst:134:broken link: https://github.com/freedomofpress/securedrop/tree/develop/docs/diagrams (429 Client Error: too many requests for url: https://github.com/freedomofpress/securedrop/tree/develop/docs/diagrams)

It might be worth adding https://github.com/freedomofpress/securedrop/tree/.* to the list.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, passing for me locally now

r"https://github.com/freedomofpress/securedrop/tree/.*",
]