Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to AniList's API v2 #535

Merged
merged 12 commits into from
Jan 21, 2018
Merged

Upgrade to AniList's API v2 #535

merged 12 commits into from
Jan 21, 2018

Conversation

salixor
Copy link
Member

@salixor salixor commented Dec 27, 2017

API V1 IS NOW DEPRECATED AND WILL SHUTDOWN MAY 1ST 2018 - USE API V2 INSTEAD
Source : documentation for AniList's API

This PR switches to the new version of the API, built on GraphQL 🚀

And it also removes the process of authentication required by the first version of the API.

Wrapper

  • Updated the wrapper in order to use the new GraphQL queries
  • Added files to store the GraphQL queries (.graphql files)
  • Removed AniList related settings since there's no longer a need for an authentication
  • Updated management/commands/anilist_tags_to_json.py accordingly (in which some elements of the wrapper are used)

Tests

  • Updated tests to reflect the change in API
  • Changed fixtures used for the tests so that they use the new API
  • Cleaned up tests that are no longer relevant
  • Added a FIXME in tests where RelatedWorks have to be built : couldn't find a way to properly mock this operation

@salixor salixor self-assigned this Dec 27, 2017
@codecov
Copy link

codecov bot commented Dec 27, 2017

Codecov Report

Merging #535 into master will decrease coverage by 3.96%.
The diff coverage is 89.34%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #535      +/-   ##
==========================================
- Coverage   75.11%   71.15%   -3.97%     
==========================================
  Files          80       80              
  Lines        4987     4933      -54     
==========================================
- Hits         3746     3510     -236     
- Misses       1241     1423     +182
Impacted Files Coverage Δ
mangaki/mangaki/settings.py 74.22% <ø> (+1.22%) ⬆️
...angaki/management/commands/anilist_tags_to_json.py 17.18% <33.33%> (+1.9%) ⬆️
mangaki/mangaki/wrappers/anilist.py 91.37% <90.36%> (-4.07%) ⬇️
mangaki/mangaki/utils/dpplib.py 10.11% <0%> (-80.37%) ⬇️
mangaki/mangaki/utils/recommendations.py 19.69% <0%> (-48.49%) ⬇️
mangaki/mangaki/utils/dpp.py 40.62% <0%> (-34.38%) ⬇️
mangaki/mangaki/algo/fit_algo.py 48.27% <0%> (-20.69%) ⬇️
mangaki/mangaki/algo/svd.py 77.35% <0%> (-15.1%) ⬇️
mangaki/mangaki/algo/knn.py 78.21% <0%> (-12.88%) ⬇️
mangaki/mangaki/algo/dataset.py 54.31% <0%> (-9.49%) ⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 908e8ee...0b93dc4. Read the comment docs.

@RaitoBezarius
Copy link
Member

🔥 🔥
I'll review it ASAP.

Copy link
Member

@RaitoBezarius RaitoBezarius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds pretty good, just few minor concerns.
That's awesome work.

raise AniListException(data['error'])
if data.get('errors'):
for error in data['errors']:
raise AniListException(error)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That means that only one error is raised. Others are silenced.
Maybe let's put an array of errors. It's up to you.

data = self._request(
'{worktype}/{id}/page',
{'worktype': worktype.value, 'id': id}
query=read_graphql_query('work-info'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could make 'work-info' as a constant static string in the head of this file, so that we can easily change all occurrences of it.

data = self._request(
'{worktype}/search/{title}',
{'worktype': worktype.value, 'title': title}
query=read_graphql_query('user-list'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same remark for 'user-list' as 'work-info'.

variables.update({'season': (season or to_anime_season(datetime.now())).name})
variables.update({'seasonYear': year or datetime.now().year})
variables.update({'status': AniListStatus.RELEASING.name} if only_airing else {})
variables.update({'perPage': 50})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it a default parameter per_page of this method.

raise RuntimeError('Malformed JSON, or AniList changed their API.')

if data['Page']['pageInfo']['hasNextPage'] and current_page < data['Page']['pageInfo']['lastPage']:
yield from self.list_seasonal_animes(year=year, season=season, only_airing=only_airing, current_page=current_page+1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

variables = {}
variables.update({'username': username})
variables.update({'mediaType': worktype.name})
variables.update({'perPage': 50})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it a default parameter.

hibike_entry = self.anilist.get_work_by_id(AniListWorks.animes, 20912)
hibike = insert_work_into_database_from_anilist(hibike_entry)
hibike_entry = self.anilist.get_work(search_id=20912)
# FIXME: properly mock the insertion of related works
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really have to mock them? Can we not add them directly? And let them aside?

@salixor salixor merged commit 138ae80 into master Jan 21, 2018
@salixor salixor deleted the mrsalixor/anilist-api-v2 branch January 21, 2018 12:27
jilljenn added a commit that referenced this pull request May 24, 2018
* Search for alternative titles and title when searching a work (#530)

* Add CFM: convex factorization machine (#512)

* Add factorization machine

* Add fastFM to requirements

* Add Cython dependency

* Add comment, remove some logs and upgrade pip

* Add python-dev and libopenblas-dev to requirements (yes yes)

* I should sleep

* Fix minor changes

* Add auto-fix, re-open, close buttons to Mangaki Fix (#527)

* Add auto-fix, re-open, close buttons to Mangaki Fix

* Fallback gracefully in case of external modifications of the state

* Mark MAL as source of truth for further MAL imports (#532)

* Add forgotten dep on fastFM in setup.py (#537)

* Removing duplicates using dedupe (#463)

* Adding management command to train dedupe

* Removing debugging and changing saving directory

* Output file now supported to retrieve clusters and scores

* adding dedupe to dev.txt, changing handler and argument sparser

* adding dedupe requirement in the right file, making some changes to dedupe.py

* Add DATA_DIR

* Up numpy

* Fix Ansible config (LOL) (#541)

* Fix stupid syntax error

* Update README

* Rewrite user profile settings (legacy routes killed, replaced by Vue.js + API) (#531)

Introduce Vue.js on settings. Kill some legacy routes. Have fun with ugly views.

* Test recommendation endpoint and saving of snapshots (#539)

* Test recommendation endpoint and saving of snapshots

* Fix annoying module problem

* Fix test

* Decode UTF-8 HTTP response

* Test DPP (#538)

* Test DPP

* Actually test DPP

* Upgrade to AniList's API v2 (#535)

* Handle AniList's API v2
* Suppress the now useless AniListRichEntry
* Get a work by ID or title in the same method
* Properly handle seasonal animes from AniList
* Update tests for the new version of the API

* Shortcuts on Mangaki (#389)

* Initial shortcuts ; proof of concept

* Plug into onchange event rather than rolling out our own custom event

* Add keyboard shortcut settings and cheatsheet

* Add migration for new profile setting

* Add new recommendation algorithm FMA (#549)

* Add new recommendation algorithm FMA

* Nit

* Add comments

* Fix hypothesis version and fix error due to upgrade (#558)

* Delete SearchIssue model

* Delete Neighbour model

* Delete Event/Attendee/Location models

* Actually delete events

* 🔫 Google Analytics

* Add new ALS (#544)

* Add new ALS

* Change nature of data

* Speed up ALS3

* Add references to algorithms

* GDPR compliancy — Step 1 (Export features) (#553)

Export features for users.

* Unreviewed: Fix setupfile → sendfile typo in setup.py

* Add X-SendFile for NGINX on Ansible playbooks (#563)

* GDPR compliancy — Step 3 (frontend and account deletion) (#564)

* MAL duplication problem test (#565)

* Improve MAL accuracy (#533)

* WIP: GDPR compliancy — Step 4 (frontend polish & copy changes) (#566)

* Frontend factor: split profile into profile-works and profile-preferences

* Fix anonymous profile view

* Prevent Flash of Uncompiled Content with the modal

* Opt-in for newsletter and research by design

* Explain use of personal data on signup page (#560)

* Add explanation on signup page

* Remove toggle from FAQ

* Fix migrations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants