[OJS] Implementing search in OJS 3

[NateWr]

The search page in OJS 3 needs two things done in time for release.

First, the search doesn't seem to work consistently. For instance, I have published articles by two authors, June T Forcht and Karen Bauman. A search for "Forcht" returns the correct article. A search for "Bauman" returns nothing. Also, although I have an article titled "Data Modelling ...", a search for "Modelling" doesn't return anything.

Second, I need to update the actual display of results as well as the list of filters. To do this, I need a sense of what filters we're planning on keeping around and implementing. Or do I remove them all? Here's a list:

• Authors
• Title
• Abstract
• Full Text
• Publication Date
• From / Until
• Index terms
• Discipline(s)
• Keyword(s)
• Type (method/approach)
• Coverage
• All index term fields

As might be expected, my preferred approach would be to have a single search field which does partial matching in any of these fields. Then only offer the most high-value filters. Off-hand, I'd say it's Authors and From/Until.

[asmecher]

Agree with your recommended approach, @NateWr. I'll also be considering whether we should simplify the search code in favour of making it very simple, with external tools e.g. Lucene/SOLR as our recommendations for anything beyond that.

[asmecher]

Does running tools/rebuildSearchIndex.php and flushing your data cache (rm -rf cache/_db/*) solve the search inconsistency? If so, then we're probably missing a call to re-index a submission upon some kind of publication event.

[NateWr]

Yes, tools/rebuildSearchIndex.php has definitely improved things. Perhaps it's because the search index hasn't been run for articles in the test data set?

I'm having trouble getting it to return more than one result. "review" is in the title of two articles but only one is returned. Just noting this for later investigation.

[asmecher]

The index should be self-maintaining (indexing required content upon publication events), and strictly speaking, it shouldn't be necessary to run the rebuildSearchIndex.php tool. Sounds like that's not the case. Noted!

[NateWr]

This is the last critical issue on my docket. I'm going to turn to working on some theming stuff in time for release. But I thought I'd make sure we're on the same page here.

My sense is that you'll get search working to your satisfaction, then I'll come in and style it and clear out some of the unused filters. But if you want me to take a more involved role here let me know. I haven't really worked with the search at all.

[asmecher]

My sense is that it's currently working, except that there may be a missing line of code to trigger re-indexing when it's needed. For now you can work around that (e.g. if search results are missing that should be there) by running the re-index script and clearing the data cache (cache/_db/*).

[asmecher]

(What I mean is: re-styling shouldn't have to wait, and I'll fix the indexing issue when I get a sec to look at it.)

[NateWr]

PRs (waiting on tests):
#1759
pkp/ojs#1001
pkp/omp#329

These PRs remove most of the old filters and tidy up the remaining ones (published before/after and authors).

I only removed the old filters from the template files. I wasn't sure if there would be desire to add them back in or what, but if not then there's some extra code in there that could be scrapped.

Here's what searching looks like in OJS.

OMP has a simpler search mechanism, but I pushed some small updates to OMP to comply with some minor template changes.

[asmecher]

Thanks, @NateWr; just a couple of comments from me and @mtub. Once those are ready, go ahead and merge. Don't worry about the JS linting problem in Travis testing, I'm working on that separately.

[NateWr]

Ok, tests are re-running. I'll update you when they're passing for final code-check and merge.

[NateWr]

@asmecher Would you be willing to look at the tests on this one when you get a chance?

OJS is still running because there was a merge conflict, but the previous tests had odd results that I couldn't figure out.
OMP seems to be timing out at odd times. I'm not really sure where to go from here with it.

[asmecher]

@NateWr, I'll watch the OJS test. Travis seems to be especially prone to race condition problems the last few days -- I suspect it's some side-effect of server load or something -- and I've had to restart tests sometimes a couple of times to get through. I've restarted your OMP tests, as I'm pretty sure they fall into that category.

[asmecher]

A little button-pressing got this one to pass tests, @NateWr. I'll go ahead and merge.

[asmecher]

Merged!

[josuevalrob]

what happened with Keywords?

[asmecher]

@josuevalrob, this issue was closed in 2016 and I'm not sure it's specifically relevant to your question -- could you post your question on the support forum, and include some more context, e.g. what version of OJS are you using, and what about keywords are you interested in?