<meta name="robots"> incorrectly set to 'noindex,follow' for Posts post type archive

[danielbachhuber]

• I've read and understood the contribution guidelines.
• I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened.

We've created a blog post archive at /blog/ following this approach of defining has_archive=>true for the post post type.

However, after we hit save for "Search Appearance Settings" (when modifying unrelated settings), the robots meta tag is incorrectly set to "noindex,follow":

Please describe what you expected to happen and why.

We'd expect the <meta name="robots"> meta tag to be absent from the page because we've checked "Yes" for the "Show the archive for Posts in search results" option:

How can we reproduce this behavior?

1. Add this code snippet to a mu-plugin or your theme's functions.php file.
2. Flush your rewrite rules.
3. Visit /blog/ and inspect source. Observe that <meta name="robots"> is absent from the page.
4. Go to "Search Appearance Settings" in the backend and hit "Save".
5. Visit /blog/ and inspect source. Observe that <meta name="robots" content="noindex,follow"/> is unexpectedly present.

Technical info

I've debugged the problem down to this line:

wordpress-seo/frontend/class-frontend.php

Line 746 in 04b6bae

if ( WPSEO_Options::get( 'noindex-ptarchive-' . $post_type, false ) ) {

This particular option returns "off" as its value, which causes "noindex" to be set for $robots['index']

$ wp shell
wp> WPSEO_Options::get( 'noindex-ptarchive-post', false )
=> string(3) "off"

I'm not familiar enough with the settings codebase to guess why "Show the archive for Posts in search results?" might be saved as "off" but happy to debug further if someone can point me in the right direction.

Used versions

• WordPress version: 5.2.1
• Yoast SEO version: 11.2.1
• Tested with theme: Twenty Nineteen

[Djennez]

Hmmm, I don't know what happens here 'exactly', but I found this:

On SEO -> Search Appearance -> Content Types -> post, there are no archive settings. Adding your code from here causes posts to be archiveable which we check for over here. This in turn adds archive settings for the post type on the aforementioned options page:

This in itself shouldn't break things, I suppose. But for some reason, the value from the slider noindex-ptarchive-post-off on the options page is saved to the database under the option of noindex-ptarchive-post.

So, at either "Yes" or "No" for the option Show the archive for Posts in search results?, the value "off" or "on" is saved to the database instead of a boolean.

As a string, by default, is a true boolean, the noindex meta is set per this condition:

wordpress-seo/frontend/class-frontend.php

Lines 746 to 748 in 04b6bae

	if ( WPSEO_Options::get( 'noindex-ptarchive-' . $post_type, false ) ) {
	$robots['index'] = 'noindex';
	}

I don't know if posts having an archive is wrong, and that's why this option in our plugin is acting strange, or if this slider is expected to work and isn't.

[danielbachhuber]

This in itself shouldn't break things, I suppose. But for some reason, the value from the slider noindex-ptarchive-post-off on the options page is saved to the database under the option of noindex-ptarchive-post.

So, at either "Yes" or "No" for the option Show the archive for Posts in search results?, the value "off" or "on" is saved to the database instead of a boolean.

Right, this was my diagnosis too.

I don't know if posts having an archive is wrong, and that's why this option in our plugin is acting strange, or if this slider is expected to work and isn't.

Posts having an archive should be fine, although it's not a default behavior for WordPress. I think something is wonky with the slider / settings page behavior.

[Djennez]

Took some time to investigate this further. I think it's this line that is causing problems:

wordpress-seo/inc/options/class-wpseo-option-titles.php

Line 273 in f25b106

if ( ! $pt->_builtin && WPSEO_Post_Type::has_archive( $pt ) ) {

Only if it's not a _builtin posttype, defaults will be created. Since Posts are a builtin posttype, these archive defaults are never created.

Since we only validate and save options after comparing them to our default values, this option never gets saved "properly"; with a boolean instead of a string.

wordpress-seo/inc/options/class-wpseo-option-titles.php

Lines 327 to 333 in f25b106

	protected function validate_option( $dirty, $clean, $old ) {
	$allowed_post_types = $this->get_allowed_post_types();
	foreach ( $clean as $key => $value ) {
	$switch_key = $this->get_switch_key( $key );
	switch ( $switch_key ) {

After removing the check for the _builtin bool, the option was successfully saved to the database.

This raises more questions for me:

• If we only save the correct type and value to the database if the enriched default exists; why was the string "off" always saved to the database?
• Is it a correct fix to remove the _builtin check. We've seen that it is possible, though unusual, to have archives for the _builtin posttypes.

Will have to check this with the devs :)

[stodorovic]

The builtin post type returns home_url (even if it's static page) as archive page if "posts" page isn't set.

Maybe it's possible to remove _the builtin check (only for 'noindex-ptarchive-' . $pt->name, but leave it for breadcrumbs). If "posts" page isn't set then default value of noindex-ptarchive-post should be true. I didn't yet check this logic. I'll try if I find spare time...

There is still breadcrumbs as an issue... This archive page will be excluded from the sitemap (because "posts" page doesn't exists), but it's possible to use filter wpseo_sitemap_post_type_archive_link to add custom URL into post sitemap.