Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Technical docs SEO guide #6077

Merged
merged 3 commits into from Aug 20, 2019
Merged

Conversation

@davidfischer
Copy link
Contributor

@davidfischer davidfischer commented Aug 16, 2019

  • This PR adds a guide about SEO for technical docs with a focus on Sphinx. It includes the basics of SEO and how RTD and Sphinx help with that.
  • A lot of this is explanations with links to existing other docs like the sitemaps guide, canonical guide, redirects, GA guide, etc. I think that's a good thing!
  • One section that I would like to add but haven't yet is a section on how good technical writing should naturally lead to better SEO. That could almost be its own guide, but I would like to include stuff like:
    • Using words and phrases that users use (and would search for) without keyword stuffing. This will help users search for stuff.
    • Writing with simpler language will also help search engines understand content.
    • Having different types of docs (how-to guides, tutorials, reference docs) and how that will cover different things people search for
@davidfischer davidfischer requested a review from Aug 16, 2019
Copy link
Member

@humitos humitos left a comment

I really like the Guide.

I left some comments about rst styling and opinions about its content structure.

Once a crawler or spider finds your site, it will follow links and redirects
in an attempt to find any and all pages on your site.
While there are a few ways to guide the search engine in its crawl
for example by using a sitemap or a robots.txt file which we'll discuss shortly,
Copy link
Member

@humitos humitos Aug 19, 2019

Not sure it worth, but we may want to link sitemap and robots.txt to each section.


Sphinx calls pages that don't have links to them "orphans"
and will throw a warning while building documentation that contains an orphan
unless the warning is silenced with the :ref:`orphan directive <sphinx:metadata>`:
Copy link
Member

@humitos humitos Aug 19, 2019

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

This is due to this:

'sphinx': ('https://sphinx.readthedocs.io/en/latest/', None),

I can test out changing it.

and will throw a warning while building documentation that contains an orphan
unless the warning is silenced with the :ref:`orphan directive <sphinx:metadata>`:

::
Copy link
Member

@humitos humitos Aug 19, 2019

Suggested change
::
.. prompt:: bash

Should disable selecting the $ and render the output accordingly.

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

I tried this but it doesn't come through cleanly:

Screen Shot 2019-08-19 at 8 53 39 AM

Copy link
Member

@humitos humitos Aug 20, 2019

Hrm... I think the $ in the commands (make) should be omitted. Although, I'm not sure how to tell what are commands and what's output.

We can leave as you wrote it for now, though.

.. sourcecode:: rst

.. image:: your-image.png
:align: right
Copy link
Member

@humitos humitos Aug 19, 2019

I think we can remove the align option from here so it's cleaner, since that option is not related with what we are trying to show in the example.

docs/guides/technical-docs-seo-guide.rst Show resolved Hide resolved
Use a robots.txt file
~~~~~~~~~~~~~~~~~~~~~

A ``robots.txt`` file is readable by crawlers
Copy link
Member

@humitos humitos Aug 19, 2019

We don't have a full Guide explaining how robots.txt works, but we could link this FAQ somewhere in this section: https://docs.readthedocs.io/en/latest/faq.html#how-can-i-avoid-search-results-having-a-deprecated-version-of-my-docs

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

I considered that and I think you're right at least for now. I think it's probably better to create a guide at some point though.

Copy link
Member

@humitos humitos Aug 20, 2019

Yes. robots.txt should have it's own guide, I agree.

When I wrote that FAQ I just wanted the minimal configuration required to live somewhere and I think it was a mistake to make it a FAQ, even with that small content should have been a Guide 😞

Copy link
Contributor Author

@davidfischer davidfischer Aug 20, 2019

We are trying to either create a new guide or spruce up one of the existing guides at least every month. That could be the next one!

unsupported versions of your documentation while keeping those docs online in case people need them.

By default, Read the Docs serves a ``robots.txt`` for you
which points to a :ref:`sitemap <guides/technical-docs-seo-guide:Use a sitemap.xml file>`
Copy link
Member

@humitos humitos Aug 19, 2019

I think I'd remove the mention to sitemap here, or I'd introduce the concept first.

When reading the guide, I felt confusing the mention to sitemap at this point without mentioning it previously or explaining anything about it.

Removing the first sentence al leaving only "To customize this file,..." works to me.

Also, the "By default," sentence can be merged into the following section (Use a sitemap.xml) where robots.txt was already presented and now we can mentioned that the default robots.txt will point to the auto-generated sitemap.


Read the Docs generates a sitemap for you that contains the last time
your documentation was updated as well as links to active versions, subprojects, and translations your project has.
We have a small separate guide on `sitemaps </guides/sitemaps>`_.
Copy link
Member

@humitos humitos Aug 19, 2019

The link does not work since it starts with /. I recommend using :ref: here.

Suggested change
We have a small separate guide on `sitemaps </guides/sitemaps>`_.
We have a small separate guide on :ref:`sitemaps <guides/sitemaps>`_.

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

It should be a :doc: link, not a ref or the broken link I did.

:figwidth: 80%
:target: ../_static/images/guides/google-search-engine-results.png

Google search engine results showing a customized meta description
Copy link
Member

@humitos humitos Aug 19, 2019

This is not a good example 😃

The description meta tag of that page is:

<meta content="How to add additional CSS stylesheets or JavaScript files to your Sphinx documentation to override your Sphinx theme or add interactivity with JavaScript." lang="en" name="description" />

but Google shows something different for some reason.

This blog post meta description does appear as Google's result description https://blog.readthedocs.com/custom-css-and-js-in-sphinx/

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

You're right. I totally missed that. I wonder why it isn't!

Copy link
Contributor Author

@davidfischer davidfischer Aug 19, 2019

It looks like Google chooses whether to use the meta description based on a few factors including the length and whether the keywords in the description match the keywords on the page. Sounds like dark arts.

Copy link
Member

@humitos humitos Aug 20, 2019

In that case, we may want to mention something "this is black magic and it may not displayed if the engine decides to not show it" :)

Otherwise, I feel that people will come back to us saying: "Hey, meta description does not work on Read the Docs" 😛

Copy link
Member

@humitos humitos Aug 20, 2019

If Google has a guide or the rules for this, we could link to that as well.

Copy link
Contributor Author

@davidfischer davidfischer Aug 20, 2019

Otherwise, I feel that people will come back to us saying: "Hey, meta description does not work on Read the Docs" 😛

I added something to that effect yesterday 😛.

Copy link
Member

@humitos humitos Aug 20, 2019

Cool! Just saw it. 👍


.. meta::
:description lang=en:
Your meta description here
Copy link
Member

@humitos humitos Aug 19, 2019

I think it's better if this example reflects what the image shows.

Following my next comment about the blog post, this rst code would look like:

.. meta::
    :description lang=en:
        Customize your documentation by adding CSS stylesheets or JavaScript files to change your docs' look and feel

davidfischer and others added 2 commits Aug 19, 2019
Co-Authored-By: Manuel Kaufmann <humitos@gmail.com>
Copy link
Member

@ericholscher ericholscher left a comment

This is 💯 content, and we should ship it ASAP :)

@@ -56,7 +56,7 @@ def get_version():
intersphinx_mapping = {
'python': ('https://python.readthedocs.io/en/latest/', None),
'django': ('https://django.readthedocs.io/en/1.11.x/', None),
Copy link
Member

@ericholscher ericholscher Aug 20, 2019

We should probably link to the canonical docs of these other two, also.

Copy link
Contributor Author

@davidfischer davidfischer Aug 20, 2019

I'll open another PR for this. This is approved so I'm going to merge it.

Copy link
Contributor Author

@davidfischer davidfischer Aug 20, 2019

@ericholscher
Copy link
Member

@ericholscher ericholscher commented Aug 20, 2019

We should also copy a lot of this content to the blog somehow :)

@davidfischer
Copy link
Contributor Author

@davidfischer davidfischer commented Aug 20, 2019

We should also copy a lot of this content to the blog somehow :)

Agreed. I was planning on a short blog post linking to this. We've had a lot of blog posts recently and a couple in the pipeline so I'm not sure on timing.

@davidfischer
Copy link
Contributor Author

@davidfischer davidfischer commented Aug 20, 2019

One section that I would like to add but haven't yet is a section on how good technical writing should naturally lead to better SEO.

I think a guide or maybe just a blog on technical writing (guides vs tutorials) as well as linking to resources about simpler language (good for SEO and translations) and other things is very good but probably not a fit in the SEO guide right now.

@davidfischer davidfischer merged commit 19cd521 into master Aug 20, 2019
3 checks passed
@davidfischer davidfischer deleted the davidfischer/technical-docs-seo-guide branch Aug 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants