Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GSoC 2018] All Search Improvements #4636

Merged
merged 86 commits into from Sep 28, 2018
Merged

[GSoC 2018] All Search Improvements #4636

merged 86 commits into from Sep 28, 2018

Conversation

@safwanrahman
Copy link
Member

@safwanrahman safwanrahman commented Sep 15, 2018

Here is the all code related to search upgrade. its rebased upon master and squashed the migration.

Things need to do

  • remove old code #4635
  • Docs of settings

This need to be merged after #4635 get merged.

@safwanrahman safwanrahman requested a review from Sep 15, 2018
@safwanrahman safwanrahman added this to Backlog in Search update via automation Sep 15, 2018
@safwanrahman safwanrahman added this to the Search improvements milestone Sep 15, 2018
@safwanrahman safwanrahman moved this from Backlog to In progress in Search update Sep 15, 2018
Copy link
Member

@ericholscher ericholscher left a comment

A few quick comments. I haven't reviewed it in full, but I think we should be pretty close to getting it shipped. 👍

.travis.yml Outdated
@@ -43,3 +43,4 @@ notifications:
branches:
only:
- master
- search_upgrade
Copy link
Member

@ericholscher ericholscher Sep 17, 2018

Can remove this.

@@ -32,18 +32,21 @@ def version_from_slug(slug, version):
return v


def find_file(filename):
def find_file(basename, pattern, path):
Copy link
Member

@ericholscher ericholscher Sep 17, 2018

Are we using this anywhere? Might be able to be deleted.

@@ -0,0 +1 @@
SEARCH_EXCLUDED_FILE = ['search.html', 'genindex.html', 'py-modindex.html']
Copy link
Member

@ericholscher ericholscher Sep 17, 2018

I think we can move this into the search.documents file, instead of needing another file for it.



class SearchPagination(PageNumberPagination):
page_size = 10
Copy link
Member

@ericholscher ericholscher Sep 17, 2018

We might want to default this higher, since we aren't paginating results on the search page. We probably want 25 here perhaps.

@safwanrahman
Copy link
Member Author

@safwanrahman safwanrahman commented Sep 20, 2018

@ericholscher Fixed as per you reviewed! I think it can be merged now!

Copy link
Member

@stsewd stsewd left a comment

I think you need to update the common submodule to pass the linter error https://github.com/rtfd/common/blob/52dfe7ab1dbda9d2a05f2d63bedac63e30a4e5ad/prospector.yml#L33-L33

Copy link
Member

@stsewd stsewd left a comment

Just noted some changes in the docs

@@ -0,0 +1,110 @@
Search
============
Copy link
Member

@stsewd stsewd Sep 21, 2018

nitpick: ====== must be the same length of the title


Read The Docs uses Elasticsearch_ instead of the built in Sphinx search for providing better search
results. Documents are indexed in the Elasticsearch index and the search is made through the API.
All the Search Code is open source and lives in the `Github Repository`_.
Copy link
Member

@stsewd stsewd Sep 21, 2018

np: Github -> GitHub


Indexing into Elasticsearch
^^^^^^^^^^^^^^^^^^^^^^^^^^^
For using search, you need to index data to the Elasticsearch Index. Run `reindex_elasticsearch`
Copy link
Member

@stsewd stsewd Sep 21, 2018

reindex_elasticsearch should be enclosed in double `

Auto Indexing
^^^^^^^^^^^^^
By default, Auto Indexing is turned off in development mode. To turn it on, change the
`ELASTICSEARCH_DSL_AUTOSYNC` settings to `True` in the `readthedocs/settings/dev.py` file.
Copy link
Member

@stsewd stsewd Sep 21, 2018

Same here, double `

~~~~~~~~~~~~~~~~~~~~~~~~~~~

After any build is successfully finished, `HTMLFile` objects are created for each of the
`HTML` files and the old version's `HTMLFile` object is deleted. By default,
Copy link
Member

@stsewd stsewd Sep 21, 2018

Same here, double `

@ericholscher ericholscher merged commit 182ab8d into master Sep 28, 2018
1 check passed
Search update automation moved this from In progress to Done Sep 28, 2018
ericholscher added a commit that referenced this issue Oct 3, 2018
This reverts commit 182ab8d, reversing
changes made to 6261427.
ericholscher added a commit that referenced this issue Oct 3, 2018
This reverts commit 182ab8d, reversing
changes made to 6261427.
agjohnson added a commit that referenced this issue Oct 3, 2018
This reverts commit 182ab8d, reversing
changes made to 6261427.
ericholscher added a commit that referenced this issue Nov 20, 2018
Squashed commit of the following:

commit b5007bb
Author: Eric Holscher <eric@ericholscher.com>
Date:   Tue Nov 20 11:41:40 2018 -0500

    Clean up HTMLFile Admin

commit 27e3dd4
Author: Eric Holscher <eric@ericholscher.com>
Date:   Mon Nov 19 15:39:20 2018 -0500

    Set replicas to 2 so we have 3 total copies

commit ee13ed6
Merge: f221f97 ef0cd7b
Author: Eric Holscher <eric@ericholscher.com>
Date:   Tue Nov 20 11:54:57 2018 -0500

    Merge remote-tracking branch 'origin/rel' into search-reapply

commit f221f97
Merge: 57c7b57 7bea6d8
Author: Eric Holscher <25510+ericholscher@users.noreply.github.com>
Date:   Mon Nov 19 15:21:00 2018 -0500

    Merge pull request #4909 from safwanrahman/search_fix

    Tuning Elasticsearch for search improvements

commit 7bea6d8
Author: Safwan Rahman <safwan.rahman15@gmail.com>
Date:   Sat Nov 17 02:17:22 2018 +0600

    more fix

commit 10a6590
Author: Safwan Rahman <safwan.rahman15@gmail.com>
Date:   Sat Nov 17 01:50:48 2018 +0600

    adding comments

commit 40865ae
Author: Safwan Rahman <safwan.rahman15@gmail.com>
Date:   Sat Nov 17 00:54:28 2018 +0600

    fixing test

commit 536874e
Author: Safwan Rahman <safwan.rahman15@gmail.com>
Date:   Sat Nov 17 00:29:34 2018 +0600

    optimize for elasticsearch

commit 57c7b57
Author: Eric Holscher <eric@ericholscher.com>
Date:   Tue Nov 6 12:10:08 2018 -0600

    Update migration

commit a1bd6a4
Merge: 95b23c2 d2137df
Author: Eric Holscher <eric@ericholscher.com>
Date:   Tue Nov 6 12:08:38 2018 -0600

    Merge branch 'master' of github.com:rtfd/readthedocs.org into search-reapply

commit d2137df
Merge: b63ef59 953a4ee
Author: Eric Holscher <25510+ericholscher@users.noreply.github.com>
Date:   Tue Nov 6 11:52:42 2018 -0600

    Merge pull request #4545 from rtfd/davidfischer/enable-timezone-support

    Enable timezone support and set timezone to UTC

commit 95b23c2
Merge: 75445f1 bc248aa
Author: Eric Holscher <25510+ericholscher@users.noreply.github.com>
Date:   Thu Nov 1 14:48:13 2018 -0500

    Merge branch 'master' into search-reapply

commit 953a4ee
Merge: 0a01b96 104c3a6
Author: Eric Holscher <eric@ericholscher.com>
Date:   Thu Nov 1 14:21:37 2018 -0500

    Merge remote-tracking branch 'origin/master' into davidfischer/enable-timezone-support

commit 75445f1
Author: Eric Holscher <eric@ericholscher.com>
Date:   Wed Oct 10 15:45:29 2018 +0200

    One more test

commit fae116c
Author: Eric Holscher <eric@ericholscher.com>
Date:   Wed Oct 10 15:45:03 2018 +0200

    Fix tests

commit 5b823ad
Author: Eric Holscher <eric@ericholscher.com>
Date:   Wed Oct 10 15:43:53 2018 +0200

    Don't change the query param, keep it q

commit 621e14f
Author: Eric Holscher <eric@ericholscher.com>
Date:   Wed Oct 10 14:12:13 2018 +0200

    Fix logic around search processing

commit 5af8307
Merge: 97e3ad8 c38efc8
Author: Eric Holscher <eric@ericholscher.com>
Date:   Wed Oct 10 12:35:16 2018 +0200

    Merge remote-tracking branch 'origin/master' into search-reapply

commit 97e3ad8
Author: Eric Holscher <eric@ericholscher.com>
Date:   Thu Oct 4 09:49:59 2018 +0200

    Revert "Revert "Merge pull request #4636 from rtfd/search_upgrade" (#4716)"

    This reverts commit 183b176.

commit 0a01b96
Author: David Fischer <david@readthedocs.org>
Date:   Tue Sep 18 15:38:00 2018 -0700

    Fix a merge fail

commit 73ad35c
Merge: ad7c76c ba34164
Author: David Fischer <david@readthedocs.org>
Date:   Tue Sep 18 15:11:48 2018 -0700

    Merge branch 'master' into davidfischer/enable-timezone-support

commit ad7c76c
Author: David Fischer <david@readthedocs.org>
Date:   Tue Aug 21 12:54:05 2018 -0700

    Found one more naive datetime

commit 46db5d9
Author: David Fischer <david@readthedocs.org>
Date:   Mon Aug 20 12:09:31 2018 -0700

    Enable timezone support and set timezone to UTC
@stsewd stsewd deleted the search_upgrade branch Feb 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Search update
  
Done
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants