Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[deviantart] can't download galleries anymore, worked fine when i first got it #5561

Closed
Shadow-night45 opened this issue May 6, 2024 · 16 comments

Comments

@Shadow-night45
Copy link

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this

[deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay.

i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

other sites that gallery-dl support it downloads just fine but not for deviantart recently, I've been trying to figure out how to fix this but can't find a solution, either there is a fix to this and I'm just blind to see it because I don't understand this program/coding stuff, or is something on deviantart end that is causing it or a recent update with gallery-dl caused something

I would like some help with this please

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Are you using your own OAuth credentials?
I can't tell, you did not provide a verbose log.

@Shadow-night45
Copy link
Author

Are you using your own OAuth credentials?
I can't tell, you did not provide a verbose log.

i don't know what that is nor do I know how to work with that and what is a verbose log?

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

Are you using your own OAuth credentials?
I can't tell, you did not provide a verbose log.

do you mean this for a verbose log? (the gallery that I'm downloading is an example of the problem I'm having)

https://imgur.com/yAHTSqB

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Almost. Verbose log means you also include the --verbose flag (or the -v short form).

But it's already obvious that you are using the built-in default credentials, which are well beyond limit already, it seems.

You need your own tokens to access the API again reliably:
https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

Almost. Verbose log means you also include the --verbose flag (or the -v short form).

But it's already obvious that you are using the built-in default credentials, which are well beyond limit already, it seems.

You need your own tokens to access the API again reliably: https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

ok I think I did all the steps in the link your provided for deviantart but I'm still getting the errors, still cant download stuff

and I'm not understanding this "verbose log"

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

The output of gallery-dl --verbose <YOUR URL HERE>

Also, please don't post screenshots or something, just paste the text output here. Fenced between triple backticks:

Three ` characters
<YOUR LOG>
Three ` characters

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

im still not understanding, because i dont understand coding stuff, all i want to do is try and fix this downloading issue with deviantart

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Then you probably did the steps you said you did wrong..

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

i dont know what i did wrong, I'm trying to get help with this because I cant figure this out on my own

all that It keeps doing is asking about (Register your own OAuth application and use its credentials to prevent this error:) then just spams (API responded with 429 Too Many Requests)

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Paste the relevant part of your config here (redact the tokens). Or the entire config.

@unmnae
Copy link

unmnae commented May 6, 2024

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this

[deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay.

i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

Try the solution from here #5558

@Shadow-night45
Copy link
Author

Paste the relevant part of your config here (redact the tokens). Or the entire config.

how do I do that?

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this
[deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay.
i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

Try the solution from here #5558

I followed everything on the configuration from this https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

the only part I'm confused on is (get a new refresh-token for the new client-id (gallery-dl oauth:deviantart)

when I do that is brings me to a new tab that says

[Your 'refresh-token' is

(the numbers and letters that I wont say)

This value has been cached and will automatically be used.]

after that I still try to do the (gallery-dl [the link of the person gallery] )

which keeps giving me what I said at the beginning

@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Okay, here's an entire config file for DeviantArt, you can copy this and use it as a starting point for your own customizations:

{
    "extractor":
    {
        "deviantart":
        {
            "gallery":
            {
                "archive-format": "g_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{author[username]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "folder":
            {
                "archive-format": "F_{folder[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{folder[owner]}", "Folders", "{folder[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "favorite":
            {
                "archive-format": "f_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "collection":
            {
                "archive-format": "C_{collection[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{collection[owner]}", "Collections", "{collection[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },

            "archive-prefix": "",
            "archive": "D:\\gallery-dl\\archive\\gallery-dl.archive.deviantart.db",

            "directory": ["DeviantArt", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
            "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}",

            "jwt": false,
            "client-id": "<YOUR_CLIENT_ID>",
            "client-secret": "<YOUR_CLIENT_SECRET>",
            "refresh-token": "<YOUR_REFRESH_TOKEN>",

            "auto-watch": false,
            "auto-unwatch": false,
            "comments": false,
            "extra": false,
            "folders": false,
            "group": true,
            "include": "gallery",
            "journals": "html",
            "metadata": true,
            "pagination": "manual",
            "public": false,

            "flat": true,
            "mature": true,
            "original": true,
            "quality": "png",

            "skip": true,
            "wait-min": 0,
            "sleep-request": [0.8, 1.2],
            "postprocessors": ["nonpicsortnontagged"]
        }
    }
}

You need to replace "client-id", "client-secret", and "refresh-token" with your own values and you're ready to go.

@Shadow-night45
Copy link
Author

Shadow-night45 commented May 6, 2024

Okay, here's an entire config file for DeviantArt, you can copy this and use it as a starting point for your own customizations:

{
    "extractor":
    {
        "deviantart":
        {
            "gallery":
            {
                "archive-format": "g_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{author[username]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "folder":
            {
                "archive-format": "F_{folder[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{folder[owner]}", "Folders", "{folder[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "favorite":
            {
                "archive-format": "f_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "collection":
            {
                "archive-format": "C_{collection[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{collection[owner]}", "Collections", "{collection[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },

            "archive-prefix": "",
            "archive": "D:\\gallery-dl\\archive\\gallery-dl.archive.deviantart.db",

            "directory": ["DeviantArt", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
            "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}",

            "jwt": false,
            "client-id": "<YOUR_CLIENT_ID>",
            "client-secret": "<YOUR_CLIENT_SECRET>",
            "refresh-token": "<YOUR_REFRESH_TOKEN>",

            "auto-watch": false,
            "auto-unwatch": false,
            "comments": false,
            "extra": false,
            "folders": false,
            "group": true,
            "include": "gallery",
            "journals": "html",
            "metadata": true,
            "pagination": "manual",
            "public": false,

            "flat": true,
            "mature": true,
            "original": true,
            "quality": "png",

            "skip": true,
            "wait-min": 0,
            "sleep-request": [0.8, 1.2],
            "postprocessors": ["nonpicsortnontagged"]
        }
    }
}

You need to replace "client-id", "client-secret", and "refresh-token" with your own values and you're ready to go.

where would I put this because I'm using notepad for the editing of this? https://github.com/mikf/gallery-dl/blob/master/docs/gallery-dl.conf but I'm not sure where I'm supposed to put it after downloading it?

and is the client-id 5-digit numbers and the client-secret 30+ digit with numbers and letters?

and is the refresh-token the one that I get from the new tab?, the one that says

(Your 'refresh-token' is

(REDACTED)

This value has been cached and will automatically be used.)

because I don't want to screw something up and end up going back to square one

@Hrxn
Copy link
Contributor

Hrxn commented May 7, 2024

Here are the config locations:

https://github.com/mikf/gallery-dl#locations

Just create one (!) file in whichever location you prefer for your config file, you can do this with notepad, or notepad++ or something.

And yes, that token from the new tab is the "refresh-token" you'll need.

@mikf mikf closed this as completed May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants