Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reply by email to comments #214

Closed
jywarren opened this issue Feb 21, 2015 · 48 comments
Closed

reply by email to comments #214

jywarren opened this issue Feb 21, 2015 · 48 comments
Assignees
@icarito @namangupta01
Labels
enhancement explains that the issue is to improve upon one of our existing features Ruby summer-of-code
Milestone
Email notificatio...

Comments

@jywarren
Copy link
Member

jywarren commented Feb 21, 2015 •
edited
Loading

For starters, here's the resource we're working from:

http://railscasts.com/episodes/313-receiving-email-with-mailman?view=asciicast

https://github.com/blog/811-reply-to-comments-from-email has some great conventions. I like how reliable and simple github email replies are.

Recommend first step could be a simple method in the Comment model, like this:

  def self.receive_mail(message)
    node_id = message.subject[/^\# (\d+)$/, 1] # this grabs the node ID from the subject line
    user_id = User.find_by(email: message.from.first)
    if node_id.present? && user_id.present?
      Comment.create! nid: node_id, body: message.body.decoded, uid: user_id
    end
  end

Then we need a mail script in the /script/ folder for something like the following:

#!/usr/bin/env ruby
require "rubygems"
require "bundler/setup"
require "mailman"

Mailman.config.pop3 = {
  server: 'pop.gmail.com', port: 995, ssl: true,
  username: ENV['GMAIL_USERNAME'],
  password: ENV['GMAIL_PASSWORD']
}

Mailman::Application.run do
  default do
    begin
      Comment.receive_mail(message)
    rescue
      Mailman.logger.error "Exception occurred while receiving message:\n#{message}"
      Mailman.logger.error [e, *e.backtrace].join("\n")
    end
  end
end

Then we'll need to contact Sebastian to try to hook this up and we won't tell anyone about the feature just yet, but can try emailing a message with subject line: Replying to #6 from an email with a corresponding registered user on PublicLab.org, and see if it appends the new comment as we've laid out, on this post, which is nid=6: https://publiclab.org/notes/warren/12-7-2010/toy-spectrometers-arrived

I might have missed something, but let's see!
@jywarren jywarren added enhancement explains that the issue is to improve upon one of our existing features bluestone labels Feb 21, 2015
@jywarren jywarren mentioned this issue Feb 21, 2015
Closed
@ebarry
Copy link
Member

ebarry commented Sep 7, 2016

Seems like this is an important functionality to have in place before we announce to the lists that we have the capability to support conversation entirely through our website.

@jywarren jywarren modified the milestone: Bluestone: migrating off google groups plan Oct 30, 2016
@aspriya
Copy link
Collaborator

aspriya commented Feb 28, 2017

@jywarren So as you have stated above, GSoC 2017 has arrived and I could be working on this. I already created a feature branch for this and doing some experiments with mailman gem. And i'm checking for other solutions that could be used also.
thanks.

@jywarren jywarren modified the milestones: Bluestone: migrating off google groups plan, Email notifications overhaul Jan 17, 2018
@jywarren jywarren added high-priority help wanted requires help by anyone willing to contribute Ruby summer-of-code labels Jan 17, 2018
@ViditChitkara
Copy link
Member

Hi @jywarren @ebarry working on it!!

@grvsachdeva
Copy link
Member

hi @ViditChitkara I guess its part of Email Integration project for this GSoC

@ViditChitkara
Copy link
Member

Yes, it is!! I have some plans about this for the soc 2018 program!!!

@grvsachdeva
Copy link
Member

Nice !!!.Actually, I have also planned to have my GSoC proposal on Email Integration Project.

@ViditChitkara
Copy link
Member

Cool!

@jywarren
Copy link
Member Author

jywarren commented Feb 15, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Jun 26, 2018 •
edited by namangupta01
Loading

This is now working! Checklist for remaining parts might be:

  • change sending email from do-not-reply@ to notifications@
  • ensure it doesn't include the reply email content (there can be a lot) below the ...
  • Extend feature to Answer comment also.
    Anything else to break out here @namangupta01 ?

@namangupta01
Copy link
Member

yes we can also figure out if we can somehow write a test for it without using email services like mail file and we can also make function test for receive_mail.

@namangupta01
Copy link
Member

namangupta01 commented Jun 27, 2018 via email

@railsprojects2018
Copy link

railsprojects2018 commented Jun 27, 2018 via email

@namangupta01
Copy link
Member

@jywarren Above message is send using outlook email id. This message was not parsed correctly. Maybe github is also parsing gmail messages?

@namangupta01
Copy link
Member

namangupta01 commented Jun 27, 2018 via email •
edited
Loading

@railsprojects2018y
Copy link

railsprojects2018y commented Jun 27, 2018 via email

@namangupta01
Copy link
Member

Above message is sent via yahoo. So the conclusion is that github only able to parse mails from gmail not other mails this may be because they are not able to trim emails from other services using any sphefic class or something like we were able to do in gmail using gmail_quote class.
I think we can also able to do parsing from yahoo by using somewhat class like yahoo_quoted but other email services provide no such points to differentiate.

@jywarren
Copy link
Member Author

jywarren commented Jun 27, 2018 via email

@namangupta01 namangupta01 self-assigned this Jun 27, 2018
@jywarren
Copy link
Member Author

jywarren commented Jul 3, 2018

So I'm not sure this is processing incoming mails right now... i checked mailman.log after publishing today and didn't see one i'd responded to arriving.

Do we have an issue open for replying to questions, and replying to answers? Questions should be pretty easy, but answers maybe need the subject line suffix (#a12345)?

@namangupta01
Copy link
Member

Can you share the logs here? I just tested here: https://publiclab.org/notes/namangupta/02-17-2018/gsoc-proposal Its working for comment. Let me look for answer.

@namangupta01
Copy link
Member

So it is working for comment i am now going to check for question and answer on local.
I have deleted the comment.
ezgif com-video-to-gif 9

@namangupta01 namangupta01 mentioned this issue Jul 4, 2018
Closed
@jywarren
Copy link
Member Author

OK - do we have reply to question working now too? Just checking!

@jywarren
Copy link
Member Author

sorry, i mean reply to question leading to a comment on the question?

@namangupta01
Copy link
Member

namangupta01 commented Jul 19, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Jul 19, 2018 via email

@namangupta01
Copy link
Member

namangupta01 commented Jul 20, 2018 via email

@namangupta01
Copy link
Member

@jywarren Both question and notes use same node model right? And reply by email is specific to node and answer so i think it should work. But i tried to comment via email to normal post even that is not comment is not added. Is the mailman server stop working after sometime or server connection get timeout after some time because when you redeploy the app it adds all the comments.

@jywarren
Copy link
Member Author

Hmm, let's ping @icarito and ask if there's more we need to do to keep the mailman server running?

@namangupta01
Copy link
Member

Can you share the mailman ending logs so that we can be fully sure about what the bug is?

@icarito
Copy link
Member

icarito commented Jul 20, 2018

sure!

icarito@rs-plots2:/srv/plots_container/plots2$ docker-compose logs mailman                                                                     │···············
Attaching to containers_mailman_1                                                                                                              │···············
mailman_1  | /usr/local/bundle/gems/activerecord-5.2.0/lib/active_record/associations/builder/collection_association.rb:27: warning: already in│···············
itialized constant TagNodeAssociationExtension                                                                                                 │···············
mailman_1  | /usr/local/bundle/gems/activerecord-5.2.0/lib/active_record/associations/builder/collection_association.rb:27: warning: previous d│···············
efinition of TagNodeAssociationExtension was here                                                                                              │···············
mailman_1  | script/mailman_server: end of file reached (EOFError)

@icarito
Copy link
Member

icarito commented Jul 20, 2018

I've restarted the service manually (with docker-compose up).
The logs added the following lines:

mailman_1  | /usr/local/bundle/gems/activerecord-5.2.0/lib/active_record/associations/builder/collection_association.rb:27: warning: already in│···············
itialized constant TagNodeAssociationExtension                                                                                                 │···············
mailman_1  | /usr/local/bundle/gems/activerecord-5.2.0/lib/active_record/associations/builder/collection_association.rb:27: warning: previous d│···············
efinition of TagNodeAssociationExtension was here

It appears to be running now!

@jywarren
Copy link
Member Author

jywarren commented Aug 4, 2018

@namangupta01 have you noticed any more outages of this feature?

What about having some place, like... publiclab.org/status, only visible to admins, where we can monitor how many emails were received and processed in the past day, past week? This could help us monitor uptime.

@namangupta01
Copy link
Member

namangupta01 commented Aug 4, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Aug 4, 2018 via email

@namangupta01
Copy link
Member

namangupta01 commented Aug 4, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Aug 4, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Aug 6, 2018

Hmm, @icarito @namangupta01 it seems like it's not running currently, I tried replying to this question:

https://publiclab.org/questions/Cindy_ExCites/08-03-2018/air-quality-balloons-an-updated-version#answer-0-comment-20293

Any chance mailman is offline again?

@jywarren jywarren removed the help wanted requires help by anyone willing to contribute label Aug 6, 2018
@namangupta01
Copy link
Member

namangupta01 commented Aug 6, 2018 via email

@icarito
Copy link
Member

icarito commented Aug 6, 2018

I made an issue about it:
#3208

Maybe it should try to relaunch when it fails? I think docker-compose has a means to restart on failure - or maybe mailman can be configured to not stop in case of error.

The trick is that it's not possible to reproduce manually because it seems to happen when GMail responds "try later".

@jywarren
Copy link
Member Author

jywarren commented Aug 6, 2018 via email

@jywarren
Copy link
Member Author

jywarren commented Dec 1, 2018

I think this is reliable enough to close up! Thanks all!

@jywarren jywarren closed this as completed Dec 1, 2018
@ghost ghost removed the in progress label Dec 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@icarito icarito

@namangupta01 namangupta01

Labels
enhancement explains that the issue is to improve upon one of our existing features Ruby summer-of-code
Projects
None yet
Milestone
Email notifications overhaul
Development

No branches or pull requests
9 participants
@jywarren @ebarry @icarito @aspriya @namangupta01 @grvsachdeva @ViditChitkara @railsprojects2018 @railsprojects2018y