[Bug] [debug] BAD BOT! [error] Failed to claim drop reward

[Kyuuso]

Describe the bug
With the latest version of the branch dev, it still seems to give an error when claiming drops.
Attached on logs whole log since started.

Logs
log-1664265690313.txt

[TychoTheTaco]

It seems Twitch is starting to implement some anti-bot checks which prevents the bot from doing some actions like claiming drops. I'm not sure how to get around this yet.

[AndyShade]

It seems Twitch is starting to implement some anti-bot checks which prevents the bot from doing some actions like claiming drops. I'm not sure how to get around this yet.

Yep they start uses https://www.kasada.io/ how i hear.

[AndyShade]

from doing some actions like claiming drops

What is mean ? Watching channels ? Or set "climed"' drops in inventory ?

[Valentin-Metz]

Have you tried "claiming" through the inventory page? - Or does that enforce the same checks?

[Kyuuso]

Have you tried "claiming" through the inventory page? - Or does that enforce the same checks?

Locally claiming them seems to work fine, even if you "script" it with a tampermonkey script like https://greasyfork.org/es/scripts/420346-auto-claim-twitch-drop .

[WrayOfSunshine]

I couldn't get the greasyfork script to work (I use greasemonkey, not tampermonkey, so that may be the issue), but there are also browser extensions that do the job.

EDIT: Disregard, multiple monitor woes.

[TychoTheTaco]

from doing some actions like claiming drops

What is mean ? Watching channels ? Or set "climed"' drops in inventory ?

From what I've seen, you can't login (unless you already have saved cookies), and you can't claim drops. Watching stream should still work.

It seems that they are somehow detecting that the bot is using Puppeteer and blocking those requests. Using a browser extension to auto claim drops might be the best alternative for now.

[WrayOfSunshine]

Currently, the bot still:

Watches streams for you (making you eligible for a drop)
Claims community points (I think)

The bot can currently not:

Claim drops for you - this is the 'claim' button on the inventory page.
You can use any number of solutions in a browser to claim these semi-automatically, but every solution I've found does require you have the drops inventory page open.

[Valentin-Metz]

The greasyfork script runs well.
No reason we can't simply put an entire non-headless firefox instance with inventory page open and script running into docker.

[AndyShade]

The greasyfork script runs well. No reason we can't simply put an entire non-headless firefox instance with inventory page open and script running into docker.

Or use puppeteer's ability load extension and use simple ChannelPointCollector. It has both chrome and firefox version.

[AndyShade]

I couldn't get the greasyfork script to work (I use greasemonkey, not tampermonkey, so that may be the issue), but there are also browser extensions that do the job.

EDIT: Disregard, multiple monitor woes.
Swith to violentmonkey - greasemonkey no support some modern and othen use js fuctions

[Luckz]

Hopefully just cosmetic, but if you manually claim, the next loop also presumes progress to be towards already accomplished goals:

[dodgex]

Maybe it is possible to include this chrome addon or talk with the dev about how he fixed his stuff?

[FelipeCalliari]

Maybe it is possible to include this chrome addon or talk with the dev about how he fixed his stuff?

Not working, you can see in the "Support" tab several people saying it is no longer claiming drops (since ~1 week ago).

[thierrybla]

Maybe it is possible to include this chrome addon or talk with the dev about how he fixed his stuff?

Not working, you can see in the "Support" tab several people saying it is no longer claiming drops (since ~1 week ago).

The extension claimed correctly for me, today. While the bot didn't.

[FelipeCalliari]

Maybe it is possible to include this chrome addon or talk with the dev about how he fixed his stuff?

Not working, you can see in the "Support" tab several people saying it is no longer claiming drops (since ~1 week ago).

The extension claimed correctly for me, today. While the bot didn't.

Hum, nice. I use Tampermonkey to do this for me and it works too (I just made a small JS script).

But, the extension worked with Puppeteer? --browser-args=["--load-extension=..."] I will give it a try.

Hello. Some time ago, Twitch introduced a new anti-bot system that caused the extension to stop working. With update 1.5.2, I was able to fully restore the extension functionality and improve its stability.

[yukaiin]

Does the bot claim the drops on the inventory page? For me the button to claim drops in the chat doesn't show up anymore.

[FelipeCalliari]

Does the bot claim the drops on the inventory page? For me the button to claim drops in the chat doesn't show up anymore.

@yukaiin I tried this extension with Puppeteer and it does not work!

However, I opened another instance of Chromium (without Puppeteer) with the extension and let a tab at https://www.twitch.tv/drops/inventory and, yes, it does claim the drop!

The only problem is Twitch might be detecting Puppeteer, so it is not as simple as simply putting this extension as a chrome addon. It's not the best solution, but I have now two instances of Chrome (one from Twitch-Drop-Bot being called by Puppeteer, and another with the extension), but, at least, it is working.

[TychoTheTaco]

I've been messing around with it a bit and it seems like their anti-bot detection is pretty complicated. I believe they are using some sort of browser fingerprinting to detect bots, maybe in addition to other factors.

If I launch a normal Chrome instance and attach to it with Puppeteer (as opposed to launching it via Puppeteer), I can successfully claim drops but it only works sometimes. Sometimes it passes the integrity check and sometimes it doesn't and I'm not sure why. The other thing to note is that it never works in headless mode so I guess they can detect that every time.

I've also tried running a non-headless Chrome instance in a docker container but I keep getting some rendering errors when trying to load the Twitch page. I tried Firefox as well, which does not have any rendering issue, but it doesn't seem to work properly with the existing code.

[WrayOfSunshine]

I think migration to Firefox for drop claiming might be a necessary switch. What a hassle, though.

[Valentin-Metz]

We could run both in parallel.

[Luckz]

I think migration to Firefox for drop claiming might be a necessary switch. What a hassle, though.

Or it's then a question of time until they also block instrumented Firefox.

[AndyShade]

TychoTheTaco any way manage FF an Chome not use puppetir ? Simple use commad line swithes for launche browsers in headles mode or hiding chrome/ff windows and set minima screen size . Manage browser via switches and devtool and set minimal resource consumptions and load extension or just use installd ?

[Luckz]

While suggesting things that are not Puppeteer, there is https://github.com/ultrafunkamsterdam/undetected-chromedriver

[Luckz]

Does the bot claim the drops on the inventory page? For me the button to claim drops in the chat doesn't show up anymore.

@yukaiin I tried this extension with Puppeteer and it does not work!

However, I opened another instance of Chromium (without Puppeteer) with the extension and let a tab at https://www.twitch.tv/drops/inventory and, yes, it does claim the drop!

I believe for me the extension works, at least with the browser (Twitch) window increased to a big enough size that it shows the chat, and not in headless mode.
I don't know if the extension claims the drops in chat, if the extension claims them when TDB opens the inventory page, or if the bigger window size and headful mode gets TDB past the bot detection.

Notes:
"--whitelisted-extension-id=kfhgpagdjjoieckminnmigmpeclkdmjm","--allowlisted-extension-id=kfhgpagdjjoieckminnmigmpeclkdmjm", I use, but I have no idea if they are useful.
--disable-extensions-except= does not disable Google Hangouts, which is the only other extension that shows up in Chrome Task Manager, so I guess it's useless? My working setup does not include this argument.

[AndyShade]

Notes: "--whitelisted-extension-id=kfhgpagdjjoieckminnmigmpeclkdmjm","--allowlisted-extension-id=kfhgpagdjjoieckminnmigmpeclkdmjm", I use, but I have no idea if they are useful. --disable-extensions-except= does not disable Google Hangouts, which is the only other extension that shows up in Chrome Task Manager, so I guess it's useless? My working setup does not include this argument.

I say trivial things but bot need to open 2 pages - a page with a video stream and inventory page from which extension claim drop. How i can see chat drop not now working in that extensions and userjs. If chorme or puppeter can execute userjs directly we may use lightweight js from greasyfork or openuserjs.org.

[Luckz]

I believe for me the extension works, at least with the browser (Twitch) window increased to a big enough size that it shows the chat, and not in headless mode.

• My account that worked in headful mode also claims new drops in headless, at "--window-size=952,512".
• The other account I am testing with did not claim drops in headless, and does not claim drops in headful at the same resolution. I think it might be be because I did not react to the super big cookie prompt on Twitch that covers most of the screen? I will let it try to go for another drop and then if it again does not claim, I will try manually dealing with the cookies prompt. I compared cookies and Twitch localstorage between the two accounts and outside of a tracking(?) cookie on the account on which I didn't react to the GDPR cookies prompt yet, it seems to be the same stuff.
• It's also possible that, because the second account I am testing has several unclaimed drops, this affects not claiming new ones. Maybe future drops won't get claimed if TDP regularly "spams" to claim the current queue. The working account (of course?) does not have that situation. Not testing this right now since I would rather not 'ruin' my test account with manual claiming, I already had to manually claim a few times to even be able to idle.

[AndyShade]

I believe for.....

If what answer for me " chat drop not working" meaning extensions or js no claim drops from chat by viewing in normal browser (no matter Chrome or FF) on usual windows size like before twitch was that anibot update(no relogins in FF).But if open inv.tab drop got clime or keep invetory tab open then they works. Of course it only behavior descriptions on me. Ah and chat allway be closed.

[Luckz]

The other account I am testing with did not claim drops in headless, and does not claim drops in headful

I think it might be be because I did not react to the super big cookie prompt on Twitch that covers most of the screen?

If you have a "What kind of cookies would you like?" prompt that refuses to disappear, or you get "Something went wrong" when you try to log in, or you cannot manually claim a drop in that browser profile with "Error Occurred: Drop was not claimed.", you are shitlisted as a bot. These are all symptoms of integrity check failure.
https://www.reddit.com/r/Twitch/comments/xld7cs/help_cookies_banner_wont_disappear_no_matter_what/
https://www.reddit.com/r/Twitch/comments/xm7how/something_went_wrong_when_trying_to_log_into/
You can verify this in the browser console too. If affected there is usually some message about integrity failure.

I spent a lot of time on trial and error to fix that troubled account. Spoilers: I think that I had ~10 pending drops was a big part of the problem.
• I cannot log in through Twitch Drops Bot.
• I can log in by hand through a regular web browser, and then hand the browser profile over to Twitch Drops Bot.
• Saving the cookies and letting Puppeteer make a fresh profile with those cookies does not work for me. I can't rule out that I was doing something wrong, but on my PC I am using a persistent browser profile and that works.
• Once Twitch Drops Bot starts, if there are pending drops to be claimed in the inventory, I believe TDB will start exhibiting bot behaviour and fail the integrity check because it tries to claim all of them at once. At least with 5–10 drops, once this happens, I can just delete the browser profile and start over. My 'fix' was to prevent execution of the relevant code:
if (isDropReadyToClaim(drop)) { in twitch_drops_bot.js

If you don't feel like replacing the cookies-accountname.json values with new ones by hand and just delete the file, you will run into trouble with TWB wanting to make you log in, which will crash due to the lack of a login prompt; an ugly way to skip it is to neuter yield this.page.focus("#login-username"); in pages\login.js. (Of course if you use the same profile every time, cookie restoring on startup isn't necessary anyway.)

After that, with a persistent browser profile created manually (using the same profile every time is not supported in the current TDB release version), and using the extension, and (I guess) with the screen resized so that the chat with drop prompt appears for the extension to claim the drops, .. it works. 😅

---

so for config.json we want:

"browser": whatever browser you made the browser profile folder with (I use Chrome & Edge)
"browser_args": here we need some things added to whatever commands you use, first add an appropriate window size, f.ex.
"--window-size=952,512"
then also load the unpacked extension, maybe it's just enough to have one of these two commands, I use both
"--disable-extensions-except=C:\\Users\\MY_FUNNY_NAME\\AppData\\Local\\Temp\\kfhgpagdjjoieckminnmigmpeclkdmjm\\1.5.2_0","--load-extension=C:\\Users\\MY_FUNNY_NAME\\AppData\\Local\\Temp\\kfhgpagdjjoieckminnmigmpeclkdmjm\\1.5.2_0"

How to specify the path to your not-yet-detected user profile is a topic for #219

To test if your profile works,
"headless": false,
Afterwards for "production" work,
"headless": "chrome",

And as a bonus, once you are happy and don't want to die of gigabyte sized log files anymore,

    "logging": {
        "level": "info"
    },

P.S.: I didn't figure out if there is a way to have Puppeteer launch Chrome without harming already installed Chrome Web Store extensions. It's of course not great to have to unpack the extension and keep it updated yourself, especially if using the same profile folder every time.

[PatrickJnr]

I guess this kinda died now :(

[miroslav-suvada]

How about adding functionality to send email when drop is ready to be claimed until you figure out some workaround?

[whyphyrl]

i can tell that this tool is working for me without docker just normal chrome and this addon https://chrome.google.com/webstore/detail/automatic-twitch-drops-mo/kfhgpagdjjoieckminnmigmpeclkdmjm

[Luckz]

i can tell that this tool is working for me without docker just normal chrome and this addon https://chrome.google.com/webstore/detail/automatic-twitch-drops-mo/kfhgpagdjjoieckminnmigmpeclkdmjm

With a secondary Chrome or the method I describe above?

[whyphyrl]

Sry im not into coding etc, xd its just a normal chrome nothing special with this addon

[JourneyOver]

Hoping this is still being looked into and hopefully a solution can be found soon..

[JourneyOver]

This isn't viable right now, but I would keep an eye on this https://github.com/Kappador/twitch-integrity at least, seems like they are trying to solve the integrity issue (though it seems they are currently sick so commits have kind of stopped for the time being) but it could be of some use in the future.

Edit: also I'm not sure how exactly, but it seems like https://github.com/DevilXD/TwitchDropsMiner got around the integrity system by using an unprotected Android app Client ID.. Maybe you could try something like that? @TychoTheTaco ----- Would obviously need some testing before releasing if it was something that is possible in this script.

[sirmaestro]

Checked on this again and went into the TwitchDropsMiner that @JourneyOver mentioned. This is the actual issue that was opened and investigated DevilXD/TwitchDropsMiner#40. I'll see if I have time to check how they did it in v13. But I can only see Twitch locking down their platform with more "extra security check" so not big hopes for the future.

[JourneyOver]

@sirmaestro this issue is currently a continuation of issue 40 seems they had locked down login a bit more as of recent but going by this comment there is somewhat of a solution in the making and he is planning on pushing something for testing when he has time.

edit: the login lockdown seems to have been somewhat limited as at least the twitchdropminer from DevilXD is back to working order in being able to login again.

[DevilXD]

Hi.

@TychoTheTaco If you're struggling with drop claiming and the integrity system, the android app client ID seems to not require having to deal with integrity at all. And this is how you can obtain an authorization token for the new ID (new ID included at the top too) in Python: https://github.com/6accOnThe6locc/twitch-mobile-token/blob/main/main.py

Caution! This linked repo is volatile and can/will disappear later on, use it while you can or save contents locally for later.

Make sure not to criss-cross the old cookie / authorization token with this new client ID for GQL usage, because Twitch is going to slap you on the wrist - you have to obtain an authorization token for the same client ID you plan on using for GQL requests later on. I'm leaving implementing all of this to you.

Hope this helps =)

[xiaodoudou]

Hi,

By removing the headless and random navigation on twitch, I was able to redeem a drop via the puppeteer instance.
Should we implement some mouse emulation and random navigation ?

Cheers,

[xiaodoudou]

Hi all,

Worked around the issue on an ugly way,
I use a VNC/Firefox docker, which I have setup Auto Twitch Drops via Greasemonkey.

You can use this container: https://hub.docker.com/r/jlesage/firefox

Cheers,

[thierrybla]

Hi all,

Worked around the issue on an ugly way, I use a VNC/Firefox docker, which I have setup Auto Twitch Drops via Greasemonkey.

You can use this container: https://hub.docker.com/r/jlesage/firefox

Cheers,

Could you explain how you did this? I can't seem to login on firefox in docker.

[xiaodoudou]

On this particular docker image?
I didnt got any issue; did tape my login/pass/2FA and done.

[thierrybla]

On this particular docker image? I didnt got any issue; did tape my login/pass/2FA and done.

Yeah with this docker, however I downloaded a cookie export extension and imported it into the browser... that works too.

[whyphyrl]

is there a docker version for chrome? the greasymonkey tool doent realy work but his addon does work very well for me but cant find a chrome docker thing sry xd not rly into docker

https://chrome.google.com/webstore/detail/automatic-twitch-drops-mo/kfhgpagdjjoieckminnmigmpeclkdmjm

[thierrybla]

On this particular docker image? I didnt got any issue; did tape my login/pass/2FA and done.

I don't know how it works for you because even claiming drops does not work in the docker.. Twitch detects it is in a docker and doesn't allow me to do anything like logging in or claiming drops.

[xiaodoudou]

I would say it is related to your cookies then or your IP maybe?

I had an issue few week ago prior to that approach that even my Chrome browser wasn't able to login.
Then I did a reset password (which is super boring to do as it will reset all your connection to the different platforms and you will need to reconnect all of them).

Since then didn't got any issue; would you mind give a try? Do you have your 2FA enable? Are you over a VPN?

On my case:

• I am constantly over a VPN
• did reset my password when Twitch did they rollout of plugin/bots detection which blocked my chrome that had greasemonkey
• do use this repo now over two docker on two account via Unraid
• do use two Firefox VNC docker via Unraid for those two accounts

Cheers,

[Chase-V]

Hi all,

Worked around the issue on an ugly way, I use a VNC/Firefox docker, which I have setup Auto Twitch Drops via Greasemonkey.

You can use this container: https://hub.docker.com/r/jlesage/firefox

Cheers,

Could you please write how to setup it? I'm just bad at linux and docker.

[dodgex]

I just got a working (non headless) bot on a linux VM by using remote debugging feature.

I also opend a feature request which contains a small how-to: #275

The bot controlled Chromium did not yet reach the point to claim a drop, but the browser instance was able to claim before. so i have hope =) I'll try to update the info if claiming the next drop was successful.

[Daivy03]

the extensions its working, you just need a workaround bc of new Chrome version. The dev of extensions in his discord said:
In Chrome version 109 was introduced a new feature/bug which boils down to the fact that as soon as you leave the tab with the muted video player, it is paused. And when you return to the tab, the playback continues.
So you don't get any rewards because you don't "watch" streams in hidden tabs.

To remedy the situation, a new experimental feature was added to the extension.
• To prevent the browser from pausing the video player, the extension will unmute it for a second (while setting the volume to minimum).
• This feature is enabled by default and can be found in the extension settings (see the screenshot): Extension Settings ➞ Prevent video stream playback from being stopped in hidden tabs (edited)