Twitter Site Change just broke Media Ripping for me

[mm125Y]

I can no longer rip twitter pages via the media tab after the changes to the site tonight:
gallery-dl --write-metadata https://twitter.com/username/media
this still works, but contains retweeted posts:
gallery-dl --write-metadata https://twitter.com/username/

Is there a workaround for this?

[biggestsonicfan]

Strange, I can't get gallery-dl https://twitter.com/username/ to work:

gallery-dl https://twitter.com/gkdlfnzo1245 --verbose
gallery-dl: Version 1.26.4-dev
gallery-dl: Python 3.11.6 - Linux-6.6.3-1-default-x86_64-with-glibc2.38
gallery-dl: requests 2.31.0 - urllib3 2.1.0
gallery-dl: Configuration Files ['${HOME}/.gallery-dl.conf']
gallery-dl: Starting DownloadJob for 'https://twitter.com/gkdlfnzo1245'
twitter: Using TwitterUserExtractor for 'https://twitter.com/gkdlfnzo1245'
twitter: Using TwitterTimelineExtractor for 'https://twitter.com/gkdlfnzo1245/timeline'
urllib3.connectionpool: Starting new HTTPS connection (1): twitter.com:443
urllib3.connectionpool: https://twitter.com:443 "GET /i/api/graphql/XA6F1nJELYg65hxOC2Ekmg/UserByScreenName?variables=%7B%22screen_name%22%3A%22gkdlfnzo1245%22%2C%22withSafetyModeUserFields%22%3Atrue%7D&features=%7B%22hidden_profile_likes_enabled%22%3Afalse%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22subscriptions_verification_info_verified_since_enabled%22%3Atrue%2C%22highlights_tweets_tab_ui_enabled%22%3Atrue%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%7D HTTP/1.1" 200 863
twitter: Sleeping 3.00 seconds (request)
urllib3.connectionpool: https://twitter.com:443 "GET /i/api/graphql/lo965xQZdN2-eSM1Jc-W_A/UserMedia?variables=%7B%22userId%22%3A%22714455845648486400%22%2C%22count%22%3A100%2C%22includePromotedContent%22%3Afalse%2C%22withClientEventToken%22%3Afalse%2C%22withBirdwatchNotes%22%3Afalse%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Atrue%7D&features=%7B%22rweb_lists_timeline_redesign_enabled%22%3Atrue%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22tweetypie_unmention_optimization_enabled%22%3Atrue%2C%22responsive_web_edit_tweet_api_enabled%22%3Atrue%2C%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Atrue%2C%22view_counts_everywhere_api_enabled%22%3Atrue%2C%22longform_notetweets_consumption_enabled%22%3Atrue%2C%22tweet_awards_web_tipping_enabled%22%3Afalse%2C%22freedom_of_speech_not_reach_fetch_enabled%22%3Atrue%2C%22standardized_nudges_misinfo%22%3Atrue%2C%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Afalse%2C%22interactive_text_enabled%22%3Atrue%2C%22responsive_web_text_conversations_enabled%22%3Afalse%2C%22longform_notetweets_rich_text_read_enabled%22%3Atrue%2C%22longform_notetweets_inline_media_enabled%22%3Afalse%2C%22responsive_web_enhance_cards_enabled%22%3Afalse%7D HTTP/1.1" 200 12439
**twitter: No results for https://twitter.com/gkdlfnzo1245/timeline**

[mikf]

Strange, I can't get gallery-dl https://twitter.com/username/ to work:

https://twitter.com/username/ uses the same endpoint as https://twitter.com/username/media by default and stops when it doesn't find anything there.

[biggestsonicfan]

Strange, I can't get gallery-dl https://twitter.com/username/ to work:

https://twitter.com/username/ uses the same endpoint as https://twitter.com/username/media by default and stops when it doesn't find anything there.

I am merely throwing into question the validity of @mm125Y statement

this still works, but contains retweeted posts:
gallery-dl --write-metadata https://twitter.com/username/

Which should not work...

[mikf]

Should maybe be fixed with c55955d.
It at least works for simple, non-R18 profiles like https://twitter.com/KirbyCafeJP/media

this still works, but contains retweeted posts:
gallery-dl --write-metadata https://twitter.com/username/

Which should not work...

It does when retweets or text-tweets are enabled, since https://twitter.com/username/ then uses the /tweets timeline.

[biggestsonicfan]

It does when retweets or text-tweets are enabled, since https://twitter.com/username/ then uses the /tweets timeline.

Ahhh interesting

[Soeroah]

I hate this so much. Why must they make it such a pain to get images.. bloody hell.

Really hope this is something that can be fixed.

Where do you do that quick dirty fix?

[Nanoka]

Where do you do that quick dirty fix?

you can install from source with python

py -3 -m pip install -U git+https://github.com/mikf/gallery-dl.git

when running from cmd it should return version 1.26.4-dev with gallery-dl --version but may conflict with exe if you have one in the same directory you run this from

the fix seems to be working as far as i have tested

[gemini0x2]

thanks @mikf for the quick fix!

[BurnBurnCB]

Where do you do that quick dirty fix?

you can install from source with python

py -3 -m pip install -U git+https://github.com/mikf/gallery-dl.git

when running from cmd it should return version 1.26.4-dev with gallery-dl --version but may conflict with exe if you have one in the same directory you run this from

the fix seems to be working as far as i have tested

Thanks for this, managed to get it and will test, although I hope the standalone release will be updated soon.

To be fair I don't dislike this update of Twitter as, following mainly artists, this way of seeing their media posts is quite better to me, if only it didn't break things...

[mikf]

I hope the standalone release will be updated soon.

You can get an executable with the latest code at https://github.com/mikf/gallery-dl/actions/runs/7151820937

[Soeroah]

Where do you do that quick dirty fix?

you can install from source with python

py -3 -m pip install -U git+https://github.com/mikf/gallery-dl.git

when running from cmd it should return version 1.26.4-dev with gallery-dl --version but may conflict with exe if you have one in the same directory you run this from

the fix seems to be working as far as i have tested

Thank you

I get an error: " ERROR: Error [WinError 2] The system cannot find the file specified while executing command git version
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?"

Is installing git another command line to run, or is it something else? I've tried running the executable mikf provided too but it just opens the CMD window and immediately closes it, nothing seems to happen, so I think I'm missing something

[mikf]

You can run the following to install from source without git

py -3 -m pip install -U https://github.com/mikf/gallery-dl/archive/master.tar.gz

I've tried running the executable mikf provided too but it just opens the CMD window and immediately closes it,

Run it from cmd.exe or PS.

[Soeroah]

You can run the following to install from source without git

py -3 -m pip install -U https://github.com/mikf/gallery-dl/archive/master.tar.gz

I've tried running the executable mikf provided too but it just opens the CMD window and immediately closes it,

Run it from cmd.exe or PS.

That appears to work (the option without git). Thank you so much

Sorry to always be asking basic questions

[a84r7a3rga76fg]

With the new media, does that mean older files can be grabbed easier?

[AAndyProgram]

Thanks @mikf for the quick fix!

[BurnBurnCB]

I hope the standalone release will be updated soon.

You can get an executable with the latest code at https://github.com/mikf/gallery-dl/actions/runs/7151820937

Oh, thanks! Somehow I had found that page yet I cIicked around and didn't realize it was a matter of... scrolling down to see the files...

Lately I've found convenient to create .bat files with the commands I use most.

[SEKCobra]

I've tried this fix, but there stimply is no output and nothing saved. Using verbose logging I see that different endpoint URLs are loaded, but nothing more.

[mikf]

With the new media, does that mean older files can be grabbed easier?

Sadly no. Twitter just slightly changed the structure of its returned data.

[kookie99]

Installing the latest dev build using the command @mikf provided above, fixed all of my Twitter related issues. Thank you for the quick and useful fix! <3

[KrisVardo]

Y'all are brilliant. New version is working great on twitter again. Thank you!