Fix issue 889 - Unicode characters not escaped

[amalajith]

Description of the Change

Add slashes to the post content to fixe the issue with Unicode characters in block attributes not escaped and getting stripped out during syndication

Closes #889

Alternate Designs

Possible Drawbacks

Verification Process

After adding the wp_slash(), the syndicated content has properly escaped slashes in the Unicode characters and the content is now properly rendered.

Checklist:

• I have read the CONTRIBUTING document.
• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my change.
• All new and existing tests passed.

Changelog Entry

Credits

Props @

[dkotter]

@amalajith Thanks for the PR! I made a few formatting changes but otherwise looks good.

But a couple things that may improve this a bit:

1. Thinking that instead of slashing the post_content in both the push and pull methods, might be better to handle that in the prepare_post utility. This would avoid duplicate code
2. The fix here only applies to Internal connections. We should probably fix this for External connections as well. Is that something you can add to this PR? Looks like a few places we would have to add that: https://github.com/10up/distributor/blob/1.6.9/includes/classes/ExternalConnections/WordPressExternalConnection.php#L701, https://github.com/10up/distributor/blob/1.6.9/includes/classes/ExternalConnections/WordPressExternalConnection.php#L1053 and https://github.com/10up/distributor/blob/1.6.9/includes/utils.php#L987
3. Is there any concern with slashing the post_content? I know that solves issues with things like unicode but just wondering if this could introduce bugs in other scenarios?

[cadic]

Just wondering, should we also wrap post_content with wp_slash for non-gutenberg content?

distributor/includes/classes/InternalConnections/NetworkSiteConnection.php

Lines 72 to 80 in aa36d6c

	$new_post_args = array(
	'post_title' => html_entity_decode( get_the_title( $post_id ), ENT_QUOTES, get_bloginfo( 'charset' ) ),
	'post_name' => $post->post_name,
	'post_content' => Utils\get_processed_content( $post->post_content ),
	'post_excerpt' => $post->post_excerpt,
	'post_type' => $post->post_type,
	'post_author' => isset( $post->post_author ) ? $post->post_author : get_current_user_id(),
	'post_status' => 'publish',
	);

[dkotter]

Just wondering, should we also wrap post_content with wp_slash for non-gutenberg content?

Yeah, I think having it for both is probably the way to go, as long as that doesn't have any unintended consequences. And as mentioned in my comment, if we move this code to the prepare_post utility method, that will take care of both Push and Pull and both block and non-block content. The one thing I just noticed here though is that the Internal Push method runs content through get_processed_content, which External connections use as well (and External connections don't use the prepare_post method). So if we add slashing to the get_processed_content method to solve External connections, we will end up slashing content twice for Internal pushes, so we'll need to handle that.

[peterwilsoncc]

Just wondering, should we also wrap post_content with wp_slash for non-gutenberg content?

Yeah, I think having it for both is probably the way to go, as long as that doesn't have any unintended consequences.

It shouldn't have any consequences, WP expects the following to be slashed and unslashes it within wp_insert_post():

• post_author
• post_date
• post_date_gmt
• post_content
• post_content_filtered
• post_title
• post_excerpt
• post_status
• post_type
• comment_status
• ping_status
• post_password
• post_name
• to_ping
• pinged
• post_modified
• post_modified_gmt
• post_parent
• menu_order
• post_mime_type
• guid

see source code

I wasn't able to reproduce the bug using external connections, so either the data is being slashed there already or this bug doesn't affect both connection types.

I was using this mini-plugin for testing -- it needs to be active on all sites & creates a test post on each site test-block.zip

[peterwilsoncc]

I've done some additional testing locally.

I think I'll move the slashing to include the entire post object rather than just the content. This will ensure the bug is fixed in it's entirety rather than risk the same bug appearing in other fields, such as title.