Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update GraphQL endpoint, support AVC1 encoded URLs, add URL detection pattern, and update RequestDetails.json #33

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

7rikazhexde
Copy link

@7rikazhexde 7rikazhexde commented Oct 9, 2023

Changes

Six changes are proposed.

  1. Change in GraphQL endpoint URL (Bqiao/twitter api change #28)
  2. Support for changed video URLs (AVC1 encoding)
  3. Addition of URL detection pattern (x)
  4. Update to RequestDetails.json (auto-update)
  5. URL detection pattern (3. commit update)
  6. Add headers to requests.get in tweet_url retrieval (statuscode 404 #32, Failed to get tweet details #34, assert html.status_code == 200 #35, Status code: 400. #36)

Details

1. Change in GraphQL endpoint URL

This is the same as #28. There was no problem with the first modification in my environment.

2. Support for changed video URLs (AVC1 encoding)

AVC1 encoding has been added to the video URL. I would like to propose a pattern that includes avc1 as a modification.
This can be used for both avc1 and non-avc1 URLs, as there are URLs that do not include avc1 in the videos posted so far.

The URL details are as follows.
7rikazhexde#4

3. Addition of URL detection pattern (x)

Since there are two URLs in the video, twitter and x, a pattern is added.

get_tweet_status_id function

Change detection pattern for status_id

get_associated_media_id function

Change detection pattern for id_str

extract_mp4s function

Change video extraction pattern (amplitude_pattern,ext_tw_pattern)

5. URL detection pattern

Allow URL patterns to be detected for both X and twitter (3. commit update).

6. Add headers to requests.get in tweet_url retrieval

The modifications are the same as the following modifications.
7rikazhexde#7

@7rikazhexde 7rikazhexde mentioned this pull request Oct 9, 2023
@7rikazhexde 7rikazhexde mentioned this pull request Jan 17, 2024
@7rikazhexde
Copy link
Author

@inteoryx
Pull requests have been updated.
Issue registrations are increasing. Please consider merge approval.

@L1ghtmann
Copy link

Bump

@yarodevuci
Copy link

Twitter URL not found error...

@7rikazhexde
Copy link
Author

The error "Failed to find main.js file" has been confirmed in issue (7rikazhexde#16), and it has been able to be fixed.

Cause
Sending a GET request with a submission page URL specification now redirects to the authentication page.

I plan to commit the same fix, but please be aware that there might be further changes to the specifications on X. (When I tried to commit the fix, there were additional changes.)

@yarodevuci
Copy link

yarodevuci commented May 19, 2024

@7rikazhexde as a walk around, I ditched the guest token and using Bearer token manually pulled from browser after Login...

@yarodevuci
Copy link

@7rikazhexde does your latest fix allows to bypass personal Bearer token ?

@7rikazhexde
Copy link
Author

@yarodevuci
Yes, with the latest changes, the code now automatically retrieves both the bearer token and the guest token. This means you no longer need to manually retrieve the bearer token from the browser after login, effectively bypassing the need for a personal bearer token.

However, if you still wish to use a manually retrieved bearer token for some reason, you can override the automatically retrieved token by replacing the following line:
bearer_token = bearer_token[0]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants