Skip to content
This repository has been archived by the owner on Feb 28, 2023. It is now read-only.

Retrieve the information about the sent / seen status #48

Closed
mmosleh opened this issue Apr 22, 2018 · 10 comments
Closed

Retrieve the information about the sent / seen status #48

mmosleh opened this issue Apr 22, 2018 · 10 comments

Comments

@mmosleh
Copy link

mmosleh commented Apr 22, 2018

I want to know how the status of the a sent message "sent" vs "seen" can be captured by the script.

Thanks!
@Mincka
Copy link
Owner

Mincka commented Apr 23, 2018

Hi,

Interesting request. I think it could be possible. May I ask for which kind of purpose you would need to know the status of the last messages you sent?

The only use case I am able to think about is some sort of mass messaging where you would like to know which users have open / seen the message.

@mmosleh
Copy link
Author

mmosleh commented Apr 23, 2018

Hi,
Thank you for your reply.

One use case is mass messaging, another is automated messaging when you want to know if the message is seen and then take some action (e.g., follow up, send another one...), and another is to have a more complete archive of your personal messages with their seen status.

I already gave it a try using your well-written code. Using the following line after reading the time stamp of the message.

ReadReceipt = dm_footer[0].cssselect('span.DMReadReceipt-statusView')

Although this variable returns the correct 'seen' or 'sent' status on the browser (using inspect), it always returns 'sent' using the script even when the message is actually seen!

I was wondering if the content receive by the script is slightly different than that of the browser. I tried to open 'https://twitter.com/messages/with/conversation?id=....' with the browser to see if the exact same content in the browser but it didn't work.

Thanks!

@Mincka
Copy link
Owner

Mincka commented Apr 24, 2018

Indeed, it seems that this is the associated text which is "Sent" by default. If you click on the blue tick, the DOM is updated with the text "Seen by everyone" / "Seen" and the "is-expanded" CSS class is added so the text is displayed.

It looks like you have to test for the presence of the CSS classes "is-seen" or "is-seenAnimated" with something like dm_footer[0].cssselect('span.is-seen'). If the array is not empty, then the message is seen (or partially seen in a group?), otherwise it is sent (or partially seen in a group?).

Seen message:

  <span class="DMReadReceipt-check is-seen is-seenAnimated">
    <span class="Icon Icon--checkLight"></span>
  </span>

Sent message:

  <span class="DMReadReceipt-check">
    <span class="Icon Icon--checkLight"></span>
  </span>

I did not see other places that could give an indication about the message status. I did not check how the JS is able to update the status. Maybe it keeps a client-side status or maybe it also relies on the detection of the CSS class. 😀

@mmosleh
Copy link
Author

mmosleh commented Apr 24, 2018

Awesome! Thanks a lot. This is exactly what I wanted. But now ran into another problem. The original script sends back the following error:

Expecting value: line 1 column 1 (char 0)

I looked at "response.content" and it returns:

b'<!DOCTYPE html>\n<html lang="en">\n<head>\n <meta charset="utf-8" />\n <title>Twitter</title>\n <style>\n body {\n background-color: #ffffff;\n }\n .link {\n color: #1da1f2;\n font-family: sans-serif;\n }\n </style>\n</head>\n<body>\n <noscript>\n \n <center><a href="/messages" class="link">Continue</a></center>\n </noscript>\n <script nonce="lYnW6pOTpZX+BQjh4Sh1/Q==">\n \n document.cookie = "app_shell_visited=1;domain=.twitter.com;path=/;max-age=30";\n \n location.replace(location.href);\n </script>\n</body>\n</html>\n'

I used the exact script from github which was working fine

Thanks!

@Mincka
Copy link
Owner

Mincka commented Apr 24, 2018 •
edited

Hum, it was working previously? It looks like a recent update to prevent parsing.

Anyway, I have a workaround. 😀

Add these few additional headers and it should make the trick. 😉
ae151b0#diff-ebdd7cd1a345766481630db935d19b6b

@Mincka Mincka changed the title sent status Retrieve the information about the sent / seen status Apr 24, 2018
@mmosleh
Copy link
Author

mmosleh commented Apr 25, 2018

Thanks a lot for updating the script and being so supportive. But still the script cannot find anything related to "seen" status. For example, the content printed by the following test line doesn't have the word "seen" for a seen message.

response=_session.get(_twitter_base_url+'/messages/with/conversation?id=xxxx',headers=_ajax_headers )
print (response.content)

While in browser without clicking on the message class="DMReadReceipt-check is-seen is-seenAnimated" exists for seen messages.

Thanks again!

@Mincka
Copy link
Owner

Mincka commented Apr 25, 2018 •
edited

You are right. I tried to find how the value was retrieved but I did not find yet.

I'm not sure this is something available in the HTML code of the tweet, because even using a browser, the "is-seen" class is not yet present in the HTTP response. It looks like it relies on another request to get this information and the DOM is updated afterward in JS.

@mmosleh
Copy link
Author

mmosleh commented Apr 26, 2018

Thanks a lot for looking into this. Unfortunately this is not available through the twitter API either. Hope I could find a solution soon.

Thanks again!

@mmosleh
Copy link
Author

mmosleh commented May 1, 2018

So I ended up resolving this using Selenium and it worked fine. Thanks for being so supportive.

@mmosleh mmosleh closed this as completed May 1, 2018
@Mincka
Copy link
Owner

Mincka commented May 1, 2018

You're welcome. Thank you for the update. I also had to use Selenium for another parsing project. It's a bit slower but more reliable. Good luck for your project.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Mincka @mmosleh