Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New mechanism for tiktok without watermark #24563

Closed
Zipdox opened this issue Apr 1, 2020 · 13 comments
Closed

New mechanism for tiktok without watermark #24563

Zipdox opened this issue Apr 1, 2020 · 13 comments

Comments

@Zipdox
Copy link

@Zipdox Zipdox commented Apr 1, 2020

I found a new mechanism for downloading tiktok videos without watermark. I wrote a proof of concept in nodejs but I'm not too experienced with python so if anyone can help implement it that'd be great.

const request = require('request-promise');

async function downloadVideo(uri){
    // User-Agent is required to fetch the right page.
    const options = {
        url: uri,
        headers: {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0'
        }
    }
    const videoPage = await request(options);
    
    // Extract JSON data from script tag.
    const regex = /<script id="__NEXT_DATA__" type="application\/json" crossorigin="anonymous">([^]*)<\/script><script crossorigin="anonymous" nomodule=/.exec(videoPage);
    if(regex == null) return console.error('Unable to extract data from page.');

    // Extract watermarked video URL after checking that the page is actually a video.
    const videoProps = JSON.parse(regex[1]);
    const videoData = videoProps.props.pageProps.videoData;
    if(videoData == undefined) return console.error('Unable to extract video data from page data.');
    const videoUrl = videoData.itemInfos.video.urls[0];

    // This is the magic trick. The video ID is hidding inside the watermarked video file. We can extract it by searching for "vid:" inside the video.
    const videoResult = await request(videoUrl);
    const position = Buffer.from(videoResult).indexOf('vid:');
    if(position == -1) return console.error('Could not find video ID in watermarked video.');
    const videoId = Buffer.from(videoResult).slice(position + 4, position + 36).toString();
    // You can retrieve the unwatermarked video with this URL and video ID.
    console.log(`https://api2.musical.ly/aweme/v1/playwm/?video_id=${videoId}`);
}

downloadVideo('https://www.tiktok.com/@user/video/id');
@dstftw dstftw closed this Apr 1, 2020
@reallyuniquename reallyuniquename mentioned this issue May 5, 2020
6 of 9 tasks complete
@nuqz
Copy link

@nuqz nuqz commented May 18, 2020

I wrote a proof of concept in nodejs but I'm not too experienced with python so if anyone can help implement it that'd be great.

@Zipdox , thank you very much. I don’t know if the python version is still needed, but just in case (skipped part with getting video URL from __NEXT_DATA__):

import io
import requests
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--video-url', type=str, dest='video_url', required=True)
args = parser.parse_args()

TIKTOK_GETTER = 'https://api2.musical.ly/aweme/v1/playwm/?video_id={video_id}'

with requests.get(args.video_url, stream=True) as response:
    response.raise_for_status()
    pos = response.content.find(b'vid:')
    bs = io.BytesIO(response.content)
    bs.seek(pos+4)
    vid = bs.read(32).decode('utf-8')

url = TIKTOK_GETTER.format(video_id=vid)
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0'
}
response = requests.head(url, headers=headers, allow_redirects=True)
print('Output: ', response.url)
@hamaz562006
Copy link

@hamaz562006 hamaz562006 commented May 20, 2020

@Zipdox , @nuqz thank you very much, how to convert this code inside the Android Code without need of any API.

@vikingUnet
Copy link

@vikingUnet vikingUnet commented Jun 18, 2020

againt not working for some videos.. url like https://api2.musical.ly/aweme/v1/playwm/?video_id={video_id} is still working for new videos, perhaps old videos moved to another data server url. Any one else decisions?

@Zipdox
Copy link
Author

@Zipdox Zipdox commented Jun 22, 2020

againt not working for some videos.. url like https://api2.musical.ly/aweme/v1/playwm/?video_id={video_id} is still working for new videos, perhaps old videos moved to another data server url. Any one else decisions?

There has been a recent change. The API URL changed and you need a user-agent to fetch the watermarkless video. I wrote a userscript to download TikTok vids without watermark.

The gist is that you fetch the watermarked video, search the file for 'vid:' and copy
Here's a breakdown of how it works:

@vikingUnet
Copy link

@vikingUnet vikingUnet commented Jun 23, 2020

Thanks, now it works :) again

@oluwabajio
Copy link

@oluwabajio oluwabajio commented Jul 10, 2020

@anthony9981
Copy link

@anthony9981 anthony9981 commented Jul 21, 2020

https://api2.musical.ly/aweme/v1/playwm/?video_id= is no longer working. Any fix

Check Zipdox comment bro: #24563 (comment)

@hamaz562006
Copy link

@hamaz562006 hamaz562006 commented Jul 31, 2020

againt not working for some videos.. url like https://api2.musical.ly/aweme/v1/playwm/?video_id={video_id} is still working for new videos, perhaps old videos moved to another data server url. Any one else decisions?

There has been a recent change. The API URL changed and you need a user-agent to fetch the watermarkless video. I wrote a userscript to download TikTok vids without watermark.

The gist is that you fetch the watermarked video, search the file for 'vid:' and copy
Here's a breakdown of how it works:

this method don't work on some videoes (secret ID can't be found) please fix

@anthony9981
Copy link

@anthony9981 anthony9981 commented Aug 5, 2020

Please provide some video link here. @hamaz562006

@hamaz562006
Copy link

@hamaz562006 hamaz562006 commented Aug 6, 2020

@anthony9981
Copy link

@anthony9981 anthony9981 commented Aug 8, 2020

@hamaz562006 Tiktok was changed json data and remove videoId in comment metadata for new uploaded videos.
Old video: https://www.metadata2go.com/result/72f49335-c9d7-45fe-a360-be17a640e0a1
New uploaded video: https://www.metadata2go.com/result/5dc10484-a729-4a55-a838-52e615078dbd
I don't see Comment with vid in new uploaded video.

@khazidhea
Copy link

@khazidhea khazidhea commented Sep 10, 2020

The "vid:" is gone from the file. Is there a new wait to get the hidden video_id? Services like https://snaptik.app/ somehow do it so there must be a way.

@Zipdox
Copy link
Author

@Zipdox Zipdox commented Sep 18, 2020

TikTok has recently stopped including it in videos. I guess they really don't like us removing watermarks. The TikTok reverse engineering guys are trying hard to figure something out without using the full mobile API but to no avail so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.