VIDCDN server is missing

[HenryQuan]

Hi, I am using the same approach as ani-cli for my own anime app and I just found out that the VIDCDN server is missing. The link to VIDCDN is the same as STREAMING. Without it, I don't think the parser will work correctly. I hope this is temporary or we need to find a new way to get the link from STREAMING.

Currently, by calling https://gogoplay1.com/encrypt-ajax.php with correct parameters can retrieve links of all MP4s and m3u8. However, there are two mystery parameters.

id: vARxWkDYfMiDYr9wKaW+3A==
token: oDLn1jQZangygjNwuGNPgw
expires: 1640403841
refer: none
ch: d41d8cd98f00b204e9800998ecf8427e <- this one
time: 79539398412775062675 <- this one

However, if ch and time can be figured out, this will be returned. MP4 links will work on itself without any headers so VLC or MPV player will be no longer needed because it can be played in a browser.

{
    "source": [
        {
            "file": "https://lb.loadfast1.com/cdn25/4ba61e475191e2f1c8759fd07e9f5e49/EP.13.v0.360p.mp4?mac=0UoKIFTrSMgVz21Ib4haHIm7wkFXVDA9ID1wjGVug4Q%3D&expiry=1640397714234",
            "label": "360 P",
            "type": "mp4"
        },
        {
            "file": "https://lb.loadfast1.com/cdn25/4ba61e475191e2f1c8759fd07e9f5e49/EP.13.v0.480p.mp4?mac=YYvOOYy5LlR9sT4xkv4ZyTtjJ1s858gTD9sP3gJgvIQ%3D&expiry=1640397714295",
            "label": "480 P",
            "type": "mp4"
        },
        {
            "file": "https://lb.loadfast1.com/cdn25/4ba61e475191e2f1c8759fd07e9f5e49/EP.13.v0.720p.mp4?mac=kNoCFLxBy2TTTXlqKWjZfdJuTzgCTkSBcEakwSyz2pQ%3D&expiry=1640397714362",
            "label": "720 P",
            "type": "mp4"
        },
        {
            "file": "https://lb.loadfast1.com/cdn25/4ba61e475191e2f1c8759fd07e9f5e49/EP.13.v0.1080p.mp4?mac=gr5VoKnc%2F%2BIjruCnJdHtSoPcUqA23zdjJknEnoELnGw%3D&expiry=1640397714417",
            "label": "1080 P",
            "type": "mp4"
        },
        {
            "file": "https://lb.loadfast1.com/cdn25/4ba61e475191e2f1c8759fd07e9f5e49/EP.13.v0.720p.mp4?mac=kNoCFLxBy2TTTXlqKWjZfdJuTzgCTkSBcEakwSyz2pQ%3D&expiry=1640397714362",
            "label": "Auto",
            "default": "true",
            "type": "mp4"
        }
    ],
    "source_bk": [
        {
            "file": "https://www15.anicdn.stream/videos/hls/zAgexKdh4SZFCn5jL12PQg/1640405145/176814/2daec4b1aa1b9266c3e7b90eef6e38e9/ep.13.1640357729.m3u8",
            "label": "hls P",
            "type": "hls"
        }
    ],
    "track": [],
    "advertising": [],
    "linkiframe": "https://sbplay2.com/e/b5mc9i9p3v1v"
}

This might be also related to #216, #215, #212, #207, #202 and #192. I hope the link can be added back soon.

[Rendevior]

I guess they implement some sort of CRSF token protections
In scraping Fullyloaded Javascript (AJAX), "Dryscrape" python library can help but i dont think they will add dryscrape to this projects because its pure Shell.
Or probably use "tenshi" #171 as temporary source.

[MeemeeLab]

They obfuscated the video player; I've reverse engineered it, And this is what I found:

Value mapping

Value1: _0x467e07
Value2: _0x1bf504
Value3: _0x35eb50
Value4: _0x4381cf
Value5: _0xacf73e
Value6: _0x304273

Parameters

id = CryptoJS.AES.encrypt(id, CryptoJS.enc.Utf8.parse(Value1), {
  iv: CryptoJS.enc.Utf8.parse(Value2)
});
time = 
      First 2: Random number (Can be any number)
      First 3~: Value2
      Final 2: Random number (Can be any number)
& Value3.substr(Value3.indexOf('&'))

Note: Requests to gogoplay requires User-Agent header, requests to encrypt-ajax.php requires User-Agent, Referer (Just a url of embedded video) and X-Requested-With (Must be 'XMLHttpRequest') headers

Calculation formulas for values

Value1:

CryptoJS.enc.Utf8.stringify(CryptoJS.AES.decrypt($('script[data-name=\x27crypto\x27]').data('value'), CryptoJS.enc.Utf8.parse(Value6.toString() + Value6.toString()), {
  iv: CryptoJS.enc.Utf8.parse(Value6)
}))

Value2: Random numbers (16 chars)

Value3: CryptoJS.enc.Utf8.stringify(Value4)

Value4:

CryptoJS.AES.decrypt(Value5, CryptoJS.enc.Utf8.parse(Value1), {
  iv: CryptoJS.enc.Utf8.parse(Value6)
})

Value5: $("[name='crypto']").attr('content')

Value6: $('script[data-name=\x27ts\x27]').data('value')

This is code I've used for my project (javascript):

function getRandomInt(min, max) {
    min = Math.ceil(min);
    max = Math.floor(max);
    return Math.floor(Math.random() * (max - min + 1)) + min;
}
function f_random(length) {
    var i = length
      , str = '';
    while (i > 0x0) {
        i--,
        str += getRandomInt(0, 9);
    }
    return str;
}

/**
 * Parses the embedded video URL to encrypt-ajax.php parameters
 * @param {cheerio} $ Cheerio object of the embedded video page
 * @param {string} id Id of the embedded video URL
 */
function generateEncryptAjaxParameters($, id) {
    const value6 = $('script[data-name=\x27ts\x27]').data('value');
    const value5 = $("[name='crypto']").attr('content');
    const value1 = 
        CryptoJS.enc.Utf8.stringify(CryptoJS.AES.decrypt($('script[data-name=\x27crypto\x27]').data('value'), CryptoJS.enc.Utf8.parse(value6.toString() + value6.toString()), {
            iv: CryptoJS.enc.Utf8.parse(value6)
        }));
    const value4 = 
        CryptoJS.AES.decrypt(value5, CryptoJS.enc.Utf8.parse(value1), {
            iv: CryptoJS.enc.Utf8.parse(value6)
        });
    const value3 = CryptoJS.enc.Utf8.stringify(value4);
    const value2 = f_random(16);
    return 'id=' + CryptoJS.AES.encrypt(id, CryptoJS.enc.Utf8.parse(value1), {
        iv: CryptoJS.enc.Utf8.parse(value2)
    }).toString() + '&time=' + '00' + value2 + '00' + value3.substring(value3.indexOf('&'));
}

I don't think this is possible on bash script though...

EDIT: My program is available at https://github.com/MeemeeLab/node-anime, Just a simple Nodejs implementation of ani-cli with obfuscation bypass.

[HenryQuan]

How about using openssl to handle AES encryption and decryption?

[alpha-hexor]

we can use a tool call phantomjs to inject the JavaScript
decrypt.js

var wpage = require('webpage').create(); 
wpage.open('{embade_url}', function(status) { 
   if (wpage.injectJs('inject.js')) { 
      var msg = wpage.evaluate(function() { 
         return testcode(); 
      }); 
      console.log(msg); 
      phantom.exit(); 
   } 
});

inject.js

function testcode(){
    const value6 = $('script[data-name=\x27ts\x27]').data('value');
    const value5 = $("[name='crypto']").attr('content');
    const value1 = CryptoJS.enc.Utf8.stringify(CryptoJS.AES.decrypt($('script[data-name=\x27crypto\x27]').data('value'), CryptoJS.enc.Utf8.parse(value6.toString() + value6.toString()), {iv: CryptoJS.enc.Utf8.parse(value6)}));
    const value4 = CryptoJS.AES.decrypt(value5, CryptoJS.enc.Utf8.parse(value1), {iv: CryptoJS.enc.Utf8.parse(value6)});
    const value3 = CryptoJS.enc.Utf8.stringify(value4);
    const value2 = f_random(16);
    return 'id=' + CryptoJS.AES.encrypt('{id from the embade url}', CryptoJS.enc.Utf8.parse(value1), {iv: CryptoJS.enc.Utf8.parse(value2)}).toString() + '&time=' + '00' + value2 + '00' + value3.substring(value3.indexOf('&'));
}

change the embade url and id in the code
Execution
phantomjs decrypt.js > link.txt
In link.txt we can see required parameters

[b31ngd3v]

@MeemeeLab thank you so so much <3

[port19x]

Closing because definitely fixed by PR 209

[port19x]

Continue discussion on #221 if necessary

[port19x]

Actually this solves so much.
On my first day as maintainer I couldn't yet comprehend what was being discussed in this thread, but now I see that this fixes not only our most prominent issue, but in the most sustainable way possible.
Implementing this solution in openssl, or an alternative cryptography library, should be our top priority and will give us a serious leg up over the gogo admins trying to screw us.
@HenryQuan @MeemeeLab @b31ngd3v @alpha-hexor

@RaynardGerraldo @CoolnsX @Dink4n

[port19x]

I'll do some groundwork with openssl tomorrow

[alpha-hexor]

openssl implementation

https://gist.github.com/alpha-hexor/a639957b054133a7d6198231dd6603f0

[port19x]

Perfect timing, looks good to me.
Only caveat: Can we do without the python -m json.tool?
It looks like that could be easily replaced with jq if I don't misunderstand its purpose.

Do you want to make a PR or should someone else integrate that solution?

[alpha-hexor]

yea jq can be used and someone else should integrate the solution as i am not good in shell scripting.

[JRCdev]

If its purpose is to just format the json output of the curl, you should be able to do the same by piping to | jq '.' ?
src: https://www.baeldung.com/linux/jq-command-json
edit: I'm also not great with bash. And while that date timestamp is pretty choice, you could get the current unix time with date +%s or printf '%(%s)T\n' -1, if that'll help blend in.

[JRCdev]

I'm working on the windows version, as someone unfamiliar with the site what's the best way to derive the embed link? Not seeing it in the html of the pages I'm currently seeing. I'm comfortable enough with the other stuff, but sed usage is also really eluding me.

[CoolnsX]

I'm working on the windows version, as someone unfamiliar with the site what's the best way to derive the embed link? Not seeing it in the html of the pages I'm currently seeing. I'm comfortable enough with the other stuff, but sed usage is also really eluding me.

actually please halt on that... because main script is not working... so it will be wastage of your time... wait till the script starts working again..

[port19x]

@CoolnsX Would you be down to integrate @alpha-hexor's solution and open a PR for it?

[b31ngd3v]

@CoolnsX Would you be down to integrate @alpha-hexor's solution and open a PR for it?

can you describe the issue? I would like to help by fixing this issue, this project helped me a lot to create my own anime streaming site ❤️

[CoolnsX]

@CoolnsX Would you be down to integrate @alpha-hexor's solution and open a PR for it?

I already dry-run it.. it was not working... so maybe it will take more time..

[port19x]

Basically, ani-cli encrypts their urls to prevent your average scraper to scrape them.
Decryption is easy and straightforward with python, but since we are a shell only project, we needed an openssl port of existing solutions.
Now we have said openssl port and am waiting for someone to integrate it into the project.
This will take some effort, as you'd have to first take a step back and search for the main gogoanime download link instead of the current sbplay and then sort of insert the decryption there.
Check the gist a few comments up in this thread for the openssl solution.
Once you ported the solution, test it with a few anime

[port19x]

@CoolnsX can you investigate why that is? And when @b31ngd3v opens his PR I need you to review it, you have rewritten the url extraction like 5 times in the past month, you know the code about as well as possible and might spot incompatibilities that I can't

[CoolnsX]

@CoolnsX can you investigate why that is? And when @b31ngd3v opens his PR I need you to review it, you have rewritten the url extraction like 5 times in the past month, you know the code about as well as possible and might spot incompatibilities that I can't

now its working I am making PR for it..
I will make it look like the previous one..

it works for some time then it prints blocked by gogoanime..

[alpha-hexor]

@CoolnsX Would you be down to integrate @alpha-hexor's solution and open a PR for it?

I already dry-run it.. it was not working... so maybe it will take more time..

Did you try it with the same embade url or different one??

[CoolnsX]

@CoolnsX Would you be down to integrate @alpha-hexor's solution and open a PR for it?

I already dry-run it.. it was not working... so maybe it will take more time..

Did you try it with the same embade url or different one??

I changed embade url every time i run it..

[alpha-hexor]

Ok . Can u share the output here

[CoolnsX]

+ ajax_url=https://gogoplay.io/encrypt-ajax.php
++ echo 'https://gogoplay.io/embedplus?id=MTc3OTkw&token=wKiClPzc2ZbJTjpJhtBzYg&expires=1642173398'
++ cut '-d?' -f2
++ cut '-d&' -f1
++ sed s/id=//g
+ video_id=MTc3OTkw
+ secret_key=3235373436353338353932393338333936373634363632383739383333323838
+ iv=34323036393133333738303038313335
++ echo -n -e MTc3OTkw
++ openssl enc -aes256 -K 3235373436353338353932393338333936373634363632383739383333323838 -iv 34323036393133333738303038313335 -a
+ ajax=Io9MuhQLU3pF3iV3SaZgeg==
++ date +%s
+ curl -H x-requested-with:XMLHttpRequest https://gogoplay.io/encrypt-ajax.php -d id=Io9MuhQLU3pF3iV3SaZgeg== -d time=1642172147
<div style='color: red;font-size: 15px;text-align: center;'>Blocked by gogoplay.io</div>

[CoolnsX]

now its working...must have been due to double quotes on every data passed to curl.. I will try to test it for couple of anime..

[alpha-hexor]

yea maybe that's the reason. Cause it works for me just fine

[alpha-hexor]

[CoolnsX]

instead of using python.. use jq -r '.source[].file'

[CoolnsX]

it will print only the links.. and install jq..

[alpha-hexor]

I am using git bash and jq doesn't come with it that's why using python

[CoolnsX]

I am using git bash and jq doesn't come with it that's why using python

here mate

[CoolnsX]

done.. made the PR.. please check it..