What does update_recent_download actually do?

[Wolfeman0101]

I'm confused what this setting actually does. Thanks.

[Avnsx]

I'm confused what this setting actually does. Thanks.

If you've used fansly scraper in the past on any creator; fansly scraper will have created a YourCreatorsName_fansly folder inside the working directory (unless you moved it out of there).

If you now launch fansly scraper let's say after a couple weeks again, because you want to update your recent download folder, for example because the creator has uploaded new content to his/her fansly; you can use update recent download to update the download folder that you've done a while ago with fansly scraper.

This mode will start hashing all recently downloaded videos and photos to be able to remember them for the duration of your scraping session. Once it finishes hashing all content, it will look through the current content of the fansly creator and compare each new content to the already known hashes (the hashes created from the in the past downloaded folder) and this way decide if the content is actually new & unique. Once it encounters too many (20%) known hashes, it will decide that it has already downloaded the entire up to date content of the fansly creator; tell you that and stop downloading.

Only if the hashes are unique, it will download the related media content from the fansly creators page, because it thinks that this content is new and unqiue.

Hope that answers your question

Edit 28 Aug 2022: Now the value for "update_recent_download" can also be set to "Auto"; it stands for Automatic when setting to this mode, hashing will always be attempted on every launch of fansly scraper but in addition fansly scraper will check by itsself if the local creator download folder already exists and update it accordingly. Only recommended on good CPUs though.

[totally-math]

Why do this hashing? It looks like you could store a little bit of metadata about each media item (like the id) and then just don't download media if the file for the given id has been downloaded already. Any reason not to do this?

[Avnsx]

Why do this hashing? It looks like you could store a little bit of metadata about each media item (like the id) and then just don't download media if the file for the given id has been downloaded already. Any reason not to do this?

There are creators out there who just re-upload the same media over and over, for example stuff like Content/Tip Menu's.
Nobody wants to view that image 5000 times in their local copy of the creators content.

[Kobalt77]

The result of this hashing process can't be stored in the folder? to have it on hand at the desired time, preventing it to be redone each time?