Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Scraper API Term & Course Search #154

Merged
merged 19 commits into from Mar 22, 2020

Conversation

ThatJuanGuy
Copy link
Collaborator

Implements api/terms and api/course/search. For api/course/search, white space between the department and course number is trimmed. Department must also be entered before course number in the search parameter.

Results are not yet filtered or sorted in any way before they are returned though
@ThatJuanGuy ThatJuanGuy added the backend Anything related to the backend API/Django label Feb 25, 2020
@gannonprudhomme gannonprudhomme changed the title Backend/scraper/api/course term Scraper API Term & Course Search Feb 25, 2020
@gannonprudhomme gannonprudhomme linked an issue Feb 25, 2020 that may be closed by this pull request
Copy link
Collaborator

@rachelconn rachelconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made quite a few comments, but most of them are just simple formatting changes/slight performance boosts. Good job on this PR, especially considering you figured out most of it by yourself

autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
autoscheduler/scraper/urls.py Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/serializers.py Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
@rachelconn
Copy link
Collaborator

See #155 and my comments on RetrieveCourseSearchView, you'll have to slightly change the way you convert the search string to handle spaces/lowercase letters. My bad for not making tests for this in the first place

Copy link
Member

@gannonprudhomme gannonprudhomme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small import fixes!

autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@rachelconn rachelconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple small changes I noticed should be made when working on my branch

autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
@gannonprudhomme
Copy link
Member

gannonprudhomme commented Mar 14, 2020

Also, given that a lot of these commits intersect with #142, I think it's best that we merge this with backend/master rather than rebasing (i.e. git merge backend/master vs git rebase backend/master). This way we only have to resolve conflicts once, rather than for each commit that conflicts with it (which is gonna be quite a few).

@ThatJuanGuy
Copy link
Collaborator Author

Remaining requested changes have been made.

Copy link
Member

@gannonprudhomme gannonprudhomme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great jobs with the tests!

Also, I got the tests setup locally and can confirm that they're passing.

I'd also like to merge backend/master into this soon, so lemme know when you're done making your next set of changes @ThatJuanGuy

autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
@ThatJuanGuy
Copy link
Collaborator Author

Kinda late but I made the latest requested changes.

@gannonprudhomme
Copy link
Member

Merged backend/master into this, so make sure you run git pull before attempting to make any further changes

Copy link
Member

@gannonprudhomme gannonprudhomme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor changes

autoscheduler/scraper/serializers.py Outdated Show resolved Hide resolved
autoscheduler/scraper/views.py Outdated Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
@ThatJuanGuy
Copy link
Collaborator Author

Latest requested changes have been made

Copy link
Collaborator

@rachelconn rachelconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some really small changes and then this looks perfect

autoscheduler/scraper/serializers.py Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
autoscheduler/scraper/tests/api_tests.py Outdated Show resolved Hide resolved
@ThatJuanGuy
Copy link
Collaborator Author

Made the changes

Copy link
Member

@gannonprudhomme gannonprudhomme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to go, great job! Ryan's gonna squash some of these commits together then merge it.

ThatJuanGuy and others added 16 commits March 22, 2020 13:53
-contains changed to startswith so that when 201 is in the 'search' parameter it doesn't return every course that takes in a 201X term instead of courses with the number 201. However, department must be inputted before the course number now.

Suggested formatting and typo fixes

Co-Authored-By: Gannon Prudhomme <gannonprudhomme@users.noreply.github.com>
Co-Authored-By: Ryan Conn <rconn478@gmail.com>
Co-Authored-By: Ryan Conn <rconn478@gmail.com>
Co-Authored-By: Gannon Prudhomme <gannonprudhomme@users.noreply.github.com>

Fixes the remaining formatting issues
Removes /api/sections and /api/course
Added test for course names including numbers

Added test for search with lowercase text
Test for "CSCE 3" now has to filter results with non-matching numbers

Added test for lowercase string with numbers ("csce 3")

Updated test for "csce" to include newly added course
Changed setUp method to setUpTestData

Changed model saving to use bulk create

Updated api/terms test to use bulk_create

Made docstrings more descriptive
This was so they could be called in api_tests.py
Also deleted some extra empty lines and added pylint silencing comments to get_desc and get_course.

Removed another empty line

Silenced parameters differ pylint warning

Fixes description for test_api_term_serializer_gives_expected_output_professional

Co-Authored-By: Gannon Prudhomme <gannonprudhomme@users.noreply.github.com>
test_api_term_serializer_handles_(un)defined_season_correctly is now called test_season_num_to_string_handles(un)defined_season_correctly to be more clear as to what is actually being tested. This was also done to campus_num_to_string related tests.
String formatting to remove spaces in search parameter is now done when search variable is assigned to rather than during the .filter in RetrieveCourseSearchView
Fixed description of some tests

The description on some tests was inacurrately referencing section serializer instead of the serializer actually beng tested
Code readability improvements

Silences "Too many public methods" pylint warning in api_tests.py

Co-Authored-By: Gannon Prudhomme <gannonprudhomme@users.noreply.github.com>

Removes unecesary department saving in api_tests.py

Code readability improvements
@rachelconn rachelconn force-pushed the backend/scraper/api/course-term branch from 3276b57 to f3397d6 Compare March 22, 2020 18:59
@rachelconn rachelconn self-requested a review March 22, 2020 19:02
@rachelconn rachelconn merged commit 88b5acc into backend/master Mar 22, 2020
@gannonprudhomme gannonprudhomme deleted the backend/scraper/api/course-term branch March 22, 2020 20:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backend Anything related to the backend API/Django
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement /api/terms and /api/course/search
4 participants