Skip to content

Full-text search support #770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
adamziel wants to merge 2 commits into from
Closed

Full-text search support #770

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0182bb3
Propose __experimentalBatchSaveEntityRecords
adamziel Nov 26, 2020
4b6ed65
Lint
adamziel Nov 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions src/wp-admin/includes/upgrade.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -878,6 +878,10 @@ function upgrade_all() {
upgrade_560();
}

if ( $wp_current_db_version < 51769 ) {
Copy link
Copy Markdown
Contributor Author

@adamziel adamziel Nov 26, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose this number arbitrarily (fulltext ticket id). We should discuss it more before merging.

Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc Nov 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's usually the SVN commit ID but your meaning is clear :) at time of writing it is 49632.

upgrade_570();
Copy link
Copy Markdown
Contributor Author

@adamziel adamziel Nov 26, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this form, it's a one-time only upgrade. Should we make it possible to retry at a later time (e.g. after migrating the site to more recent MySQL version)? If yes, what would be the best approach to do that?

}

maybe_disable_link_manager();

maybe_disable_automattic_widgets();
Expand Down Expand Up @@ -2275,6 +2279,35 @@ function upgrade_560() {
save_mod_rewrite_rules();
}
}
/**
* Executes changes made in WordPress 5.7.0.
*
* @ignore
* @since 5.7.0
*/
function upgrade_570() {
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc Nov 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excluding the option I think the rest can be done by updating schema.php

global $wp_current_db_version, $wpdb;

if ( $wp_current_db_version < 51769 ) {
/*
* Create fulltext indexes for `wp_posts`.
*/
$result = $wpdb->query( "ALTER TABLE $wpdb->posts ADD FULLTEXT INDEX `wp_posts_fulltext` (`post_title` ASC, `post_excerpt` ASC, `post_content` ASC);" );
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc Dec 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nicer way of detecting this that can be added to the $wpdb->has_cap() method would be dandy.

// Return early if fulltext indexes are unsupported
if ( ! $result || is_wp_error( $result ) ) {
return;
}
$wpdb->query( "ALTER TABLE $wpdb->posts ADD FULLTEXT INDEX `wp_posts_fulltext_title` (`post_title` ASC);" );
$wpdb->query( "ALTER TABLE $wpdb->posts ADD FULLTEXT INDEX `wp_posts_fulltext_excerpt` (`post_excerpt` ASC);" );
$wpdb->query( "ALTER TABLE $wpdb->posts ADD FULLTEXT INDEX `wp_posts_fulltext_content` (`post_content` ASC);" );

/*
* When upgrading from WP < 5.7.0 enable fulltext search.
*/
update_option( 'fulltext_search_available', '1' );
update_option( 'fulltext_search_enabled', '1' );
}
}

/**
* Executes network-level upgrade routines.
Expand Down
10 changes: 10 additions & 0 deletions src/wp-admin/options-reading.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,16 @@
</fieldset></td>
</tr>

<?php if ( wp_fulltext_search_available() ) : ?>
<tr class="option-site-visibility">
<th scope="row"><?php _e( 'Full-text search' ); ?> </th>
<td><fieldset><legend class="screen-reader-text"><span><?php _e( 'Full-text search' ); ?> </span></legend>
<label for="fulltext_search_enabled"><input name="fulltext_search_enabled" type="checkbox" id="fulltext_search_enabled" value="1" <?php echo checked( '1', get_option( 'fulltext_search_enabled' ) ); ?> />
<?php _e( 'Use full-text search on this site' ); ?></label>
</fieldset></td>
</tr>
<?php endif ?>

Comment on lines +211 to +220
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc Dec 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My preference would be to go without an option. If the indexes are always created then I am not sure what the gain is in giving admins options they may not understand.

Is there a common reason an owner might want to turn it off if it's available on their server?

Copy link
Copy Markdown
Member

@TimothyBJacobs TimothyBJacobs Dec 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah +1 to not making this a UI option.

<?php do_settings_fields( 'reading', 'default' ); ?>
</table>

Expand Down
1 change: 1 addition & 0 deletions src/wp-admin/options.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@
'page_on_front',
'page_for_posts',
'blog_public',
'fulltext_search_enabled',
),
'writing' => array(
'default_category',
Expand Down
72 changes: 52 additions & 20 deletions src/wp-includes/class-wp-query.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1373,26 +1373,49 @@ protected function parse_search( &$q ) {
*/
$exclusion_prefix = apply_filters( 'wp_query_search_exclusion_prefix', '-' );

foreach ( $q['search_terms'] as $term ) {
// If there is an $exclusion_prefix, terms prefixed with it should be excluded.
$exclude = $exclusion_prefix && ( substr( $term, 0, 1 ) === $exclusion_prefix );
if ( $exclude ) {
$like_op = 'NOT LIKE';
$andor_op = 'AND';
$term = substr( $term, 1 );
} else {
$like_op = 'LIKE';
$andor_op = 'OR';
}
if ( wp_fulltext_search_enabled() && empty( $q['exact'] ) ) {
$fulltext_terms = array();
$positive_terms = array();

if ( $n && ! $exclude ) {
$like = '%' . $wpdb->esc_like( $term ) . '%';
$q['search_orderby_title'][] = $wpdb->prepare( "{$wpdb->posts}.post_title LIKE %s", $like );
foreach ( $q['search_terms'] as $term ) {
if ( $exclusion_prefix && ( substr( $term, 0, 1 ) === $exclusion_prefix ) ) {
$term = '-' . substr( $term, 1 );
} else {
$positive_terms[] = $term;
}
$fulltext_terms[] = $term;
}
$fulltext_query = implode( ' ', $fulltext_terms );
$q['search_orderby_title'][] = $wpdb->prepare(
"MATCH({$wpdb->posts}.post_title) AGAINST (%s IN BOOLEAN MODE)",
implode( ' ', $positive_terms )
);
$search .= $wpdb->prepare(
"{$searchand}(MATCH ({$wpdb->posts}.post_title, {$wpdb->posts}.post_excerpt, {$wpdb->posts}.post_content) AGAINST (%s IN BOOLEAN MODE))",
$fulltext_query
);
} else {
foreach ( $q['search_terms'] as $term ) {
// If there is an $exclusion_prefix, terms prefixed with it should be excluded.
$exclude = $exclusion_prefix && ( substr( $term, 0, 1 ) === $exclusion_prefix );
if ( $exclude ) {
$like_op = 'NOT LIKE';
$andor_op = 'AND';
$term = substr( $term, 1 );
} else {
$like_op = 'LIKE';
$andor_op = 'OR';
}

if ( $n && ! $exclude ) {
$like = '%' . $wpdb->esc_like( $term ) . '%';
$q['search_orderby_title'][] = $wpdb->prepare( "{$wpdb->posts}.post_title LIKE %s", $like );
}

$like = $n . $wpdb->esc_like( $term ) . $n;
$search .= $wpdb->prepare( "{$searchand}(({$wpdb->posts}.post_title $like_op %s) $andor_op ({$wpdb->posts}.post_excerpt $like_op %s) $andor_op ({$wpdb->posts}.post_content $like_op %s))", $like, $like, $like );
$searchand = ' AND ';
$like = $n . $wpdb->esc_like( $term ) . $n;
$search .= $wpdb->prepare( "{$searchand}(({$wpdb->posts}.post_title $like_op %s) $andor_op ({$wpdb->posts}.post_excerpt $like_op %s) $andor_op ({$wpdb->posts}.post_content $like_op %s))", $like, $like, $like );
$searchand = ' AND ';
}
}

if ( ! empty( $search ) ) {
Expand Down Expand Up @@ -1516,7 +1539,11 @@ protected function parse_search_order( &$q ) {

// Sentence match in 'post_title'.
if ( $like ) {
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_title LIKE %s THEN 1 ", $like );
if ( wp_fulltext_search_enabled() ) {
$search_orderby .= $wpdb->prepare( "WHEN MATCH({$wpdb->posts}.post_title) AGAINST (%s IN BOOLEAN MODE) THEN 1 ", $q['s'] );
} else {
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_title LIKE %s THEN 1 ", $like );
Copy link
Copy Markdown
Contributor Author

@adamziel adamziel Nov 26, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mimics LIKE behavior 1:1 for now. I believe MATCH() relevancy score would be a good fit here, maybe it could even replace WHEN/THEN entirely.

}
}

// Sanity limit, sort as sentence when more than 6 terms
Expand All @@ -1532,8 +1559,13 @@ protected function parse_search_order( &$q ) {

// Sentence match in 'post_content' and 'post_excerpt'.
if ( $like ) {
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_excerpt LIKE %s THEN 4 ", $like );
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_content LIKE %s THEN 5 ", $like );
if ( wp_fulltext_search_enabled() ) {
$search_orderby .= $wpdb->prepare( "WHEN MATCH({$wpdb->posts}.post_excerpt) AGAINST (%s IN BOOLEAN MODE) THEN 4 ", $q['s'] );
$search_orderby .= $wpdb->prepare( "WHEN MATCH({$wpdb->posts}.post_content) AGAINST (%s IN BOOLEAN MODE) THEN 5 ", $q['s'] );
} else {
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_excerpt LIKE %s THEN 4 ", $like );
$search_orderby .= $wpdb->prepare( "WHEN {$wpdb->posts}.post_content LIKE %s THEN 5 ", $like );
}
}

if ( $search_orderby ) {
Expand Down
26 changes: 26 additions & 0 deletions src/wp-includes/functions.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7781,3 +7781,29 @@ function is_php_version_compatible( $required ) {
function wp_fuzzy_number_match( $expected, $actual, $precision = 1 ) {
return abs( (float) $expected - (float) $actual ) <= $precision;
}

/**
* Check if fulltext search is available.
*
* @since 5.7.0
*
* @return bool Whether fulltext search is available.
*/
function wp_fulltext_search_available() {
$is_available = get_option( 'fulltext_search_available' ) === '1';
return apply_filters( 'fulltext_search_available', $is_available );
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc Dec 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Docblock on filters (but can wait as the ticket is discussed, tbh)

}

/**
* Check if fulltext search is enabled.
*
* Returns false if it's enabled but not available.
*
* @since 5.7.0
*
* @return bool Whether fulltext search is enabled.
*/
function wp_fulltext_search_enabled() {
$is_enabled = get_option( 'fulltext_search_enabled' ) === '1';
return wp_fulltext_search_available() && apply_filters( 'fulltext_search_enabled', $is_enabled );
}
2 changes: 1 addition & 1 deletion src/wp-includes/version.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
*
* @global int $wp_db_version
*/
$wp_db_version = 49632;
$wp_db_version = 51769;

/**
* Holds the TinyMCE version.
Expand Down