Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#161967026 Add Custom Search Functionality #24

Merged
merged 1 commit into from
Dec 13, 2018
Merged

#161967026 Add Custom Search Functionality #24

merged 1 commit into from
Dec 13, 2018

Conversation

babbageLabs
Copy link
Contributor

@babbageLabs babbageLabs commented Dec 12, 2018 •
edited

What does this PR do?

Add the search and filter functionality

Description of Task to be completed?

update requirements.txt with search engine requirements by introducing Django-filter, haystack and whoosh. This provides two search approaches ie filtering results using Django-filter and building and searching the index using haystack and whoosh.

Filter

GET /api/articles/q/filter/< ?param=value&param2=value >

The filter backend provides filter by the following parameters:

  • title
  • title__contains
  • description
  • description__contains
  • slug
  • slug__contains
  • body
  • body__contains
  • taglist
  • taglist__contains
  • createdAt
  • createdAt__year__lt
  • createdAt__year__gt
  • updatedAt
  • updatedAt__year__lt
  • updatedAt__year__gt
  • author__username
Search

GET /api/articles/q/search/?

The search endpoint employs the Whoosh backend to index the articles to provide
faster search and more search parameters:

  • author
  • title
  • tags
  • keywords
  • pub_date
  • edit_date
  • slug

all the above parameters take the following additional options as
parameter__option or parameter__not_option

    content
    contains
    exact
    gt
    gte
    lt
    lte
    in
    startswith
    endswith
    range
    fuzzy

How should this be manually tested?

  1. git clone https://github.com/andela/ah-jumanji.git
  2. cd ah-jumanji
  3. git checkout ft-add-search-filtering-functionality-161967026
  4. Setup a virtual environment and activate it
  5. pip install -r requirements.txt
  6. python manage.py makemigrations
  7. python manage.py migrate
  8. python manage.py runserver
  9. pytest -v -k "TestSearchFilter"
    -- optionally to update the search index
  10. ./manage.py update_index --settings=authors/settings

Any background context you want to provide?

The search method employed offers a search engine to index the articles and improve search efficiency . This is faster and more scalable. The choice of whoosh is informed by its pure python nature hence it does not require additional dependencies.

The search index is updated in real-time as the database is updated. This can how be made more efficient by employing a key value memory store like Redis mq.

What are the relevant pivotal tracker stories?

#161967026

Screenshots (if appropriate)

image

image

Checklist:

  • My code follows the style guidelines of this project
  • At least 2 people have reviewed my PR
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • My PR has one commit.

houndci-bot
houndci-bot reviewed Dec 12, 2018
View reviewed changes
authors/apps/search/models.py Outdated
@@ -0,0 +1,2 @@


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blank line at end of file

authors/apps/search/admin.py Outdated
@@ -0,0 +1 @@

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blank line at end of file

authors/settings.py Outdated
# is updated
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blank line at end of file

authors/apps/articles/serializers.py Outdated
@@ -1,10 +1,11 @@
from rest_framework import serializers

from authors.apps.authentication.serializers import UserSerializer
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'authors.apps.authentication.serializers.UserSerializer' imported but unused

@babbageLabs babbageLabs force-pushed the ft-add-search-filtering-functionality-161967026 branch from fd14f3a to 8a815aa Compare December 12, 2018 15:36
@babbageLabs babbageLabs changed the title feat(add-custom-search-functionality): Set up search/filtering #161967026 Add-custom-search-functionality Dec 12, 2018
@ghost ghost changed the title #161967026 Add-custom-search-functionality #161967026 Add Custom Search Functionality Dec 13, 2018
@babbageLabs babbageLabs force-pushed the ft-add-search-filtering-functionality-161967026 branch from 8a815aa to 0beb9d8 Compare December 13, 2018 14:16
GransonO
GransonO approved these changes Dec 13, 2018
View reviewed changes
Copy link
Contributor

@GransonO GransonO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well illustrated and to the point. Works as expected. LGTM

ghost
ghost approved these changes Dec 13, 2018
View reviewed changes
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would have been great if we had a test that verifies that searching for a particular item will return that item.

mirr254
mirr254 approved these changes Dec 13, 2018
View reviewed changes
Copy link
Contributor

@mirr254 mirr254 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@babbageLabs
feat(add-custom-search-functionality): Set up search/filtering
e1042f1 
- write unit tests for search endpoints
- update requirements with search dependencies
- add search application with a search index,filter ans serializer
- configure search backend indexing in settings.py
- update the documentation

[Finishes #161967026]
@babbageLabs babbageLabs force-pushed the ft-add-search-filtering-functionality-161967026 branch from 0beb9d8 to e1042f1 Compare December 13, 2018 16:45
@mirr254 mirr254 merged commit 15abf81 into develop Dec 13, 2018
@ghost ghost deleted the ft-add-search-filtering-functionality-161967026 branch December 17, 2018 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@houndci-bot houndci-bot houndci-bot left review comments

@mirr254 mirr254 mirr254 approved these changes

@GransonO GransonO GransonO approved these changes

@EmmanuelBeja EmmanuelBeja Awaiting requested review from EmmanuelBeja

@fabzer0 fabzer0 Awaiting requested review from fabzer0

@gwako94 gwako94 Awaiting requested review from gwako94

@mashafrancis mashafrancis Awaiting requested review from mashafrancis

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@babbageLabs @houndci-bot @mirr254 @GransonO