Everything broke AGAIN

[TheFrenchGhosty]

Everything broke AGAIN.

@pablouser1 I asked you to test your code here, you told me that you would, you didn't, so things broke AGAIN.

This is the last time I'm reporting a bug, next time I'm killing my instance, I'm sick of babysitting it because every single commit break something. https://github.com/pablouser1/ProxiTok/issues?q=is%3Aissue+author%3ATheFrenchGhosty

On some profiles:

Warning: Undefined property: stdClass::$detail in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 23

Fatal error: Uncaught TypeError: TikScraper\Models\Info::setDetail(): Argument #1 ($detail) must be of type object, null given, called in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 23 and defined in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php:13 Stack trace: #0 /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php(23): TikScraper\Models\Info->setDetail(NULL) #1 /var/www/html/vendor/pablouser1/tikscraper/src/Cache.php(41): TikScraper\Models\Info->fromCache(Object(stdClass)) #2 /var/www/html/vendor/pablouser1/tikscraper/src/Items/Base.php(32): TikScraper\Cache->handleInfo('user-bianca2849') #3 /var/www/html/vendor/pablouser1/tikscraper/src/Items/User.php(14): TikScraper\Items\Base->__construct('bianca2849', 'user', Object(TikScraper\Sender), Object(TikScraper\Cache), false) #4 /var/www/html/vendor/pablouser1/tikscraper/src/Api.php(25): TikScraper\Items\User->__construct('bianca2849', Object(TikScraper\Sender), Object(TikScraper\Cache), false) #5 /var/www/html/app/Controllers/UserController.php(15): TikScraper\Api->user('bianca2849') #6 [internal function]: App\Controllers\UserController::get('bianca2849') #7 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(476): forward_static_call_array(Array, Array) #8 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(440): Bramus\Router\Router->invoke('UserController@...', Array) #9 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(287): Bramus\Router\Router->handle(Array, true) #10 /var/www/html/index.php(17): Bramus\Router\Router->run() #11 {main} thrown in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 13
`

On other profiles:

Fatal error: Uncaught Error: Typed property TikScraper\Models\Info::$detail must not be accessed before initialization in /var/www/html/vendor/pablouser1/tikscraper/src/Items/User.php:52 Stack trace: #0 /var/www/html/vendor/pablouser1/tikscraper/src/Items/User.php(43): TikScraper\Items\User->feedStandard(0) #1 /var/www/html/app/Controllers/UserController.php(16): TikScraper\Items\User->feed(0) #2 [internal function]: App\Controllers\UserController::get('willsmith') #3 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(476): forward_static_call_array(Array, Array) #4 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(440): Bramus\Router\Router->invoke('UserController@...', Array) #5 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(287): Bramus\Router\Router->handle(Array, true) #6 /var/www/html/index.php(17): Bramus\Router\Router->run() #7 {main} thrown in /var/www/html/vendor/pablouser1/tikscraper/src/Items/User.php on line 52

[pablouser1]

This issue has nothing to do with the latest commits...

e1fbf75 fixed an issue with video cache
c041e84 fixed an issue with Video playback on Safari

None of them modified the code that is currently throwing those errors. Everyting was working fine until today.

I think that TikTok is starting to enforce more their "bot detection" mecanism on their web. Normally, when you were detected as a bot, you were required to type a captcha. That initial request would throw a 403 code error, which then the program would correctly handle.

That doesn't happen anymore, now it returns a 200 OK code, which is not expected. That's what's causing those errors. I'm looking for a fix now.

In the meantime, it looks like other wrapper for the TikTok Api are also having the same issue. So maybe TikTok has implemented more anti-bot tactics?

[pablouser1]

Hey @TheFrenchGhosty I've checked both your instance and my local docker deployment and it looks like it got somehow fixed, without doing anything?

What was happening (I think) was that TikTok asked for a captcha if you entered https://www.tiktok.com/@USERNAME_HERE without entering the main site (https://www.tiktok.com) first.

Maybe they thought it was too strict and stopped it? Or maybe it was only a mistake on their end...
I'll wait some days to see if this comes back, I'll also add a method to detect captchas even if they send a 200 HTTP code, just in case

[TheFrenchGhosty]

@pablouser1

This issue has nothing to do with the latest commits...

Sorry, I assumed it was since every other time it was because of the latest commits... I was kinda angry because I'm really starting to get sick of babysitting a service I don't use at all, as I said in my Matrix room:

The most unstable software I run:

• Massive amount of resource required to run it (because it runs a full browser) though it's better now that he made a dedicated signer, before that it needed a full browser that used 2-5 full cores and multiples gigabytes of ram
• Hours spent to make my own docker image
• Hours spend to debug/fix it when it breaks

I've been running an instance for less than 2 months: it broke 4 or 5 times.
It's ALWAYS caused by [your] untested code

All this work is for something I DON'T USE!
I spent HOURS for a software I NEVER use

[pablouser1]

Sorry, I assumed it was since every other time it was because of the latest commits... I was kinda angry because I'm really starting to get sick of babysitting a service I don't use at all

Don't worry, that already happened multiple times so I understand that you thought those commits caused it.

Massive amount of resource required to run it (because it runs a full browser) though it's better now that he made a dedicated signer, before that it needed a full browser that used 2-5 full cores and multiples gigabytes of ram

Yeah, the chromedriver signing was really painful, it was also the main reason I couldn't use standard mode on the official instance. I started using it because every TikTok Wrapper used this method. At least the new signer is (way) more resource friendly than a whole Chrome instance.

It's ALWAYS caused by [your] untested code

I now test everything on Docker before pushing so this shouldn't happen again. This is basically the first time I develop something that is being used by more people other than me, so I was used to test everything using my local installation only.

[EsmailELBoBDev2]

sad to say it but my instance is broken too, mostly 403 error code? (https://proxitok.esmailelbob.xyz/@wwe)

and here logs from docker compose:

Attaching to proxitok-web, proxitok-signer, proxitok-redis
proxitok-signer | 2022-06-21T02:52:11.952964375Z App listening on port: 8080
proxitok-web | 2022-06-21T02:52:12.994894060Z AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.29.0.4. Set the 'ServerName' directive globally to suppress this message
proxitok-redis | 2022-06-21T02:52:00.639779261Z 1:C 21 Jun 2022 02:52:00.639 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
proxitok-redis | 2022-06-21T02:52:00.639807636Z 1:C 21 Jun 2022 02:52:00.639 # Redis version=7.0.0, bits=64, commit=00000000, modified=0, pid=1, just started
proxitok-redis | 2022-06-21T02:52:00.639812601Z 1:C 21 Jun 2022 02:52:00.639 # Configuration loaded
proxitok-web | 2022-06-21T02:52:13.010025310Z AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.29.0.4. Set the 'ServerName' directive globally to suppress this message
proxitok-web | 2022-06-21T02:52:13.170281295Z [Tue Jun 21 02:52:13.170166 2022] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.53 (Debian) PHP/8.1.6 configured -- resuming normal operations
proxitok-web | 2022-06-21T02:52:13.170306690Z [Tue Jun 21 02:52:13.170206 2022] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
proxitok-redis | 2022-06-21T02:52:00.640692350Z 1:M 21 Jun 2022 02:52:00.640 # Server initialized
proxitok-redis | 2022-06-21T02:52:00.640705551Z 1:M 21 Jun 2022 02:52:00.640 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
proxitok-web | 2022-06-21T02:52:16.004149713Z 172.29.0.1 - - [21/Jun/2022:02:52:15 +0000] "GET / HTTP/1.0" 200 1196 "-" "Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0"
proxitok-web | 2022-06-21T02:52:18.351871077Z 172.29.0.1 - - [21/Jun/2022:02:52:18 +0000] "GET /redirect?term=wwe&type=user HTTP/1.0" 302 211 "https://proxitok.esmailelbob.xyz/" "Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0"
proxitok-web | 2022-06-21T02:52:18.585575291Z 172.29.0.1 - - [21/Jun/2022:02:52:18 +0000] "GET /@wwe HTTP/1.0" 403 1892 "https://proxitok.esmailelbob.xyz/" "Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0"

[EsmailELBoBDev2]

oh nevermind it works now 👍

I did not notice that legacy mode is dead 😢

[TheFrenchGhosty]

@pablouser1 I updated... it's broken AGAIN!

https://proxitok.pussthecat.org/@billieeilish

Warning: Undefined property: stdClass::$detail in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 23

Fatal error: Uncaught TypeError: TikScraper\Models\Info::setDetail(): Argument #1 ($detail) must be of type object, null given, called in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 23 and defined in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php:13 Stack trace: #0 /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php(23): TikScraper\Models\Info->setDetail(NULL) #1 /var/www/html/vendor/pablouser1/tikscraper/src/Cache.php(41): TikScraper\Models\Info->fromCache(Object(stdClass)) #2 /var/www/html/vendor/pablouser1/tikscraper/src/Items/Base.php(31): TikScraper\Cache->handleInfo('user-billieeili...') #3 /var/www/html/vendor/pablouser1/tikscraper/src/Items/User.php(13): TikScraper\Items\Base->__construct('billieeilish', 'user', Object(TikScraper\Sender), Object(TikScraper\Cache)) #4 /var/www/html/vendor/pablouser1/tikscraper/src/Api.php(23): TikScraper\Items\User->__construct('billieeilish', Object(TikScraper\Sender), Object(TikScraper\Cache)) #5 /var/www/html/app/Controllers/UserController.php(15): TikScraper\Api->user('billieeilish') #6 [internal function]: App\Controllers\UserController::get('billieeilish') #7 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(476): forward_static_call_array(Array, Array) #8 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(440): Bramus\Router\Router->invoke('UserController@...', Array) #9 /var/www/html/vendor/bramus/router/src/Bramus/Router/Router.php(287): Bramus\Router\Router->handle(Array, true) #10 /var/www/html/index.php(17): Bramus\Router\Router->run() #11 {main} thrown in /var/www/html/vendor/pablouser1/tikscraper/src/Models/Info.php on line 13

or:

https://proxitok.pussthecat.org/@willsmith

There was an error processing your request!
HTTP Code: 200
API error code -1 (Unknown error)

It worked 10 before minutes on an older version...

[pablouser1]

I think I figured it out... The program was storing on cache an invalid request (-1 error code) which crashes when accessed later because all cache items should only have valid requests. It should be fixed now, make sure to wipe your Redis cache and tell me if it goes OK.

BTW, all those crashes are related to rate limits. I'm not able to trigger them locally, so I'm just guessing what is wrong from the logs. I'm also having those issues on the official instance but only when I have higher traffic.

[TheFrenchGhosty]

@pablouser1 Problem is still here.

[EsmailELBoBDev2]

@TheFrenchGhosty works with me https://proxitok.esmailelbob.xyz/@wwe