[Feature request] Use Anime download history to download new episodes

[Quicksilver151]

store anime and episode number of anime users have downloaded.
have an extra function that checks all the anime in the history files for new episodes and downloads new available episodes for all anime in history files

[sdaqo]

What exactly do you mean by "new episodes"? Like new episodes that air?

[Quicksilver151]

so basically
the program checks the anime listed in the history and then reqests the website to return the current number of available episodes for that anime
then it compares the number of episodes available to the number of episodes downloaded
and if the number of episodes available are more, then it downloads them

this is useful since i watch atleast like 10 anime every season
then i can just run this code once a week and get all the updates without having to search for and download them all individually

[Quicksilver151]

the idea is inspired by this manga downloader which... unfortunately didnt really work but i made a fork of it with a quick fix
https://github.com/Quicksilver151/Manga-Downloader

[sdaqo]

then it compares the number of episodes available to the number of episodes downloaded
and if the number of episodes available are more, then it downloads them

That wont work since you will have a problem when you want to download only a specific episode, it will also download that.

How about a kind of menu where you could mark all anime that you want to be downloaded when running the function you requested for, and it saves them in a separate file? Then, when running the function, it bulk-downloads the newest episodes that are available on gogoanime. Sounds good to you?

[Quicksilver151]

That wont work since you will have a problem when you want to download only a specific episode, it will also download that.

not sure what you mean by that. what exactly is the problem youre describing here? I dont quite get it

in any case.. if you didnt understand what i wanted for this feature, then check out that one repo i posted above

How about a kind of menu where you could mark all anime that you want to be downloaded when running the function you requested for, and it saves them in a separate file? Then, when running the function, it bulk-downloads the newest episodes that are available on gogoanime. Sounds good to you?

im not sure about that
instead of marking them, shouldnt it just be possible to store the names of all the anime in a file and use those
then we wont have to mark all the anime manually
or am i misunderstanding this?

[sdaqo]

not sure what you mean by that. what exactly is the problem youre describing here? I dont quite get it
in any case.. if you didnt understand what i wanted for this feature, then check out that one repo i posted above

Well I didnt quite sayed that right, I mean if you search the history of dowloaded animes there will be some you dont want to download, but those will also be download then.

instead of marking them, shouldnt it just be possible to store the names of all the anime in a file and use those

Maybe mark is not the right word but yes thats what I mean, but you cant just store the names, there has to be some kind of link, if you look whats saved in the history.txt file you will find something like this: https://gogoanime.wiki/death-note-episode-1, so you have to either store these manually or, we could implement a function where you can query gogoanime and then it automatically stores something like this: https://gogoanime.wiki/category/slow-loop. Then if you run the bulk downloader or whatever we will call it and it reads the file and downloads all the latest episodes.

[Quicksilver151]

Well I didnt quite sayed that right, I mean if you search the history of dowloaded animes there will be some you dont want to download, but those will also be download then.

ah that makes sense

also one thing we could do to fix that is to automatically remove anime that have the status 'competed' on it from the download history list
since this it only makes sense that its gonna be used for ongoing anime

implement a function where you can query gogoanime and then it automatically stores something like this: https://gogoanime.film/category/slow-loop. Then if you run the bulk downloader or whatever we will call it and it reads the file and downloads all the latest episodes.

yea that sounds good

[sdaqo]

Ok, nice I will see when I have time, but this sounds fun. And something I also would want to add is that you can not only download them but also binge watch them, this is a feature also coming soon made by @Dankni95 in #18.

[sdaqo]

Hey @Quicksilver151 I implemented the feature request now in 9d31411. Could you test it if everything works for you?

[Quicksilver151]

It works
this is now my favorite anime downloaded
long live python webscrapers

but in anycase
why the progress bar isnt blue any more
i liked the previous colors

[sdaqo]

We can make it blue if you think it looks better.

[Quicksilver151]

wait wait wai

Seasonal save Path: /media/renderinguser/PC Disk EXT/anipy-cli/user_files/seasonals.txt
Downloads are stored in: /media/renderinguser/PC Disk EXT/anipy-cli/download
[a] Add Anime
[e] Delete one anime from seasonals
[l] List animes in seasonals file
[d] Download newest episodes
[w] Binge watch newest episodes
[q] Quit
Enter Option: d
Already downloaded or watched: 
-> genjitsu-shugi-yuusha-no-oukoku-saikenki-part-2-episode-1
-> shingeki-no-kyojin-the-final-season-part-2-episode-1
-> sono-bisque-doll-wa-koi-wo-suru-episode-1
Do you want to exclude these from downloading? (y/N): y
Downloading: 
Do you want to download these/this episode(s)? (y/N): y
 /m/r/P/anipy-cli    ./main.py -sq 480                                                                                                                                     23.3s
***Seasonal Mode***
Seasonal save Path: /media/renderinguser/PC Disk EXT/anipy-cli/user_files/seasonals.txt
Downloads are stored in: /media/renderinguser/PC Disk EXT/anipy-cli/download
[a] Add Anime
[e] Delete one anime from seasonals
[l] List animes in seasonals file
[d] Download newest episodes
[w] Binge watch newest episodes
[q] Quit
Enter Option: d
No animes to download, add some.
Enter Option:

why is it made in a way that the anime is removed when downloading if no new episodes are available
shouldnt it just skip downloading them if new episodes arent available

i think its best for users to manually remove anime from the list
the auto removal opition should be only if anime has status:completed on it
i think thatll make it better

also instead of storing the link to the latest episode, you could instead store just a number

[sdaqo]

Hey I guessed that there were going to be bugs, could you send steps to reproduce? I will have time later that day.

[Quicksilver151]

its not a bug
its just the way you made it to work

Already downloaded or watched: 
-> shingeki-no-kyojin-the-final-season-part-2-episode-1
-> genjitsu-shugi-yuusha-no-oukoku-saikenki-part-2-episode-1
-> sono-bisque-doll-wa-koi-wo-suru-episode-1
Do you want to exclude these from downloading? (y/N):

i just want this part to not exist basically ^
(edit: or.. make it so that that menu only appears if that anime has status:completed on it)

it will just cancel download automatically when it sees that here are no more updates

[sdaqo]

Oh I see but actually it gets the newest links and compares it to the ones that you last watched so when there airs a new episode they shouldn't be there.

[Quicksilver151]

oh
i see
well in that case id appreciate if that gets fixed

[sdaqo]

Hey fixed this now, the problem was that when you have nothing to download/watch it will wipe the file since it has nothing to write, I made it so that if the episode list is empty you get put back in the menu.

[Quicksilver151]

wai
it used to wipe the file?

actually that wasnt the issue i was having
ok so..

suppose i downloaded episode one of aot with the seasonals command
then the seasonals history contains something like

https://gogoanime.film/category/shingeki-no-kyojin-the-final-season-part-2#https://gogoanime.film/shingeki-no-kyojin-the-final-season-part-2-episode-1

then, when running the seasonal command, it would attempt to download episode one again
even tho episode one is already downloaded
would it be possible to make it so that it ignores downloading unless there is a new episode?

[sdaqo]

There should be a prompt saying something like this: Do you want to exclude these from downloading? (y/N): then you press y or Y and it dowsnt include it when downloading. If you mean that I should fetch the status of the anime, well I could o that but that just overccompliefies the script. Let me try to explain roughly whats going on: it fetches the newest episode link then compares it to the link that is stored after the "#", if they match this episode will appear in the prompt I just wrote about, you can either exclude it or include it when downloading. It then downloads the episode if it didnt got excluded and updates the link after the "#" with the one that was just being downloaded. I build it that way to prevent somethhing, let me explain: lets say you got anime A and C, anime A airs on fridays and C on saturdays, now forr instance lets say its friday, you start the script and it will download ep 1 of anime A then on saturday you statrt it again because you want anime C but now it will attempt to download A and C, you dont want that though so there will be a prompt asking to exclude anime A.

[Quicksilver151]

There should be a prompt saying something like this: Do you want to exclude these from downloading? (y/N):

inputing y there clears the history file...
wait, is that the bug you mensioned that you fixed?
oh lemmie check real quick

[Quicksilver151]

ok so in any case
got a few more bugs to work around
2 bugs and one suggestion is included here
bug1:
if i add for example, attack on titan season 2 to seasonals with the program, i am then unable to add season 1 or 3
seems like the program cant differentiate between seasons
steps to reproduce:

• add attack on titan season 2 to seasonals.txt with the program
• add season 1 of it as well
• see that there is only season 2 added

bug2:
it seems to only download the latest episodes of a given anime
which means if i stay 2 weeks without using this program to download anime, the next time run it to download seasonals, it would miss an episode
i suppose there really is no other way of doing this other than requesting gogoanime to return the number of episodes in a given anime for each anime in seasonals.txt

suggestion:
also. instead of storing links for both the anime and the episode, wouldnt it be better to like store the link to the anime and then just store a number next to it to tell what episode was dowloaded last?

[Quicksilver151]

and as for this:

lets say you got anime A and C, anime A airs on fridays and C on saturdays, now forr instance lets say its friday, you start the script and it will download ep 1 of anime A then on saturday you statrt it again because you want anime C but now it will attempt to download A and C, you dont want that though so there will be a prompt asking to exclude anime A.

if you could just fetch the available episode number for each anime. Then on Saturday, only anime C will be downloaded
because using the episode list, the program will know that A didnt get an update. only C did

check out this if you need any reference to how it can work
https://github.com/Quicksilver151/Manga-Downloader
^thats a manga downloader i mensioned earlier in this thread
i use it to download updates for like over 50 manga at this point and it downloads the latest chapters of the manga whenever available

[sdaqo]

bug1: you are right thats odd will have to look into it
bug2: never thought of that but yyou are right. How about it stores for instance ep. 1 and than after two weeks it will see on gogoanime that the anime is on ep. 3 and than download the difference of the two numbers so 2.
suggestion: could do that but that wont improve anything, link is way more convinient

if you could just fetch the available episode number for each anime. Then on Saturday, only anime C will be downloaded
because using the episode list, the program will know that A didnt get an update. only C did

Thats exactly what the script does.

check out this if you need any reference to how it can work
https://github.com/Quicksilver151/Manga-Downloader

The suggestion for bug2 should do that.

[Quicksilver151]

How about it stores for instance ep. 1 and than after two weeks it will see on gogoanime that the anime is on ep. 3 and than download the difference of the two numbers so 2.

exactly that

[sdaqo]

Ok nice, we found a solution 👌 Will implement it when I have time.

[Quicksilver151]

just one more thing
could you make Y the default choice for both

Already downloaded or watched: 
-> shingeki-no-kyojin-the-final-season-part-2-episode-1
-> genjitsu-shugi-yuusha-no-oukoku-saikenki-part-2-episode-1
-> sono-bisque-doll-wa-koi-wo-suru-episode-1
Do you want to exclude these from downloading? (y/N): 
Downloading: 
-> shingeki-no-kyojin-the-final-season-part-2-episode-1
-> genjitsu-shugi-yuusha-no-oukoku-saikenki-part-2-episode-1
-> sono-bisque-doll-wa-koi-wo-suru-episode-1
-> sabikui-bisco-episode-1
Do you want to download these/this episode(s)? (y/N): 
Ok exiting now

that would make more sense since choosing y for both would be the more common choice in normal usage

[sdaqo]

Yeah will do.

[sdaqo]

@Quicksilver151 Done!
Stuff I did:

• It now downloads all episodes in the range off last watched and new
• The exclude prompt is now gone, since its not necessary anymore, it just goes back to the menu if there are no episode to download/watch
• Download bar is now blue in seasonal mode
• The default for the confirm is now y
• when adding anime to seasonal mode it will ask you for the last watched episode (0 if nothing has been watched)

[Quicksilver151]

eh?
i get this error when selecting download in seasonal mode now

Enter Option: d
Traceback (most recent call last):
  File "/media/renderinguser/PC Disk EXT/anipy-cli/main.py", line 122, in <module>
    seasonal.main_activity()
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 331, in main_activity
    menu() 
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 306, in menu
    start_action(False)
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 182, in start_action
    episode_urls = compare()
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 170, in compare
    range_list = list(range(int(i), int(j)))
ValueError: invalid literal for int() with base 10: ''

edit: seems like this is due to the previous way of storing anime in the list

[Quicksilver151]

and adding episode number is also buggy in seasonal

[14] shingeki-no-kyojin-season-3-part-2-dub
[15] shingeki-no-kyojin-the-final-season-dub
[16] shingeki-no-kyojin-the-final-season-part-2
[17] shingeki-no-kyojin-movie-1-guren-no-yumiya
[18] shingeki-no-kyojin-movie-2-jiyuu-no-tsubasa
Enter Number: 16
Last episode you watched (0 to start from the beginning) [1-1]: 1
Invalid number

[Quicksilver151]

also
seems like having issues when there are extra lines in the seasonals.txt

Enter Option: d
Traceback (most recent call last):
  File "/media/renderinguser/PC Disk EXT/anipy-cli/main.py", line 122, in <module>
    seasonal.main_activity()
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 331, in main_activity
    menu() 
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 306, in menu
    start_action(False)
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 182, in start_action
    episode_urls = compare()
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 164, in compare
    i = get_newest_episodes(i) 
  File "/media/renderinguser/PC Disk EXT/anipy-cli/src/seasonal.py", line 124, in get_newest_episodes
    querys = requests.get(url)
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/sessions.py", line 456, in prepare_request
    p.prepare(
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/models.py", line 316, in prepare
    self.prepare_url(url, params)
  File "/home/renderinguser/.local/lib/python3.8/site-packages/requests/models.py", line 390, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?

took me a while to figure out what was causing this lmao

i guess the seasonals.txt is not meant to be edited manually
so the only real bug here is being unable to add anime with 1 episode available like mensioned in the comment above

[sdaqo]

Ah I see, I forgot one '=' should work now