Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Fixed IMAP support, added specs and functional test, refactored code. #46

Merged
merged 14 commits into from

2 participants

@ndbroadbent

Hey,

I've tidied up the IMAP pull request (#18):

  • added mocks, specs, and functional test
  • refactored the code that POP3 and IMAP share
  • organized spec_helper a bit, using a spec/support directory

All specs are passing

Cheers :)

lib/mailman/receiver/imap.rb
((10 lines not shown))
+
+ # @param [Hash] options the receiver options
+ # @option options [MessageProcessor] :processor the processor to pass new
+ # messages to
+ # @option options [String] :server the server to connect to
+ # @option options [Integer] :port the port to connect to
+ # @option options [String] :username the username to authenticate with
+ # @option options [String] :password the password to authenticate with
+ def initialize(options)
+ @processor, @username, @password, @server, @filter, @port = nil, nil, nil, nil, ["NEW"], 143
+
+ @processor = options[:processor] if options.has_key? :processor
+ @username = options[:username] if options.has_key? :username
+ @password = options[:password] if options.has_key? :password
+ @filter = options[:filter] if options.has_key? :filter
+ @port = options[:port] if options.has_key? :port
@titanous Owner

Lines 19-25 can be refactored significantly:

 @processor = options[:processor]
 @username  = options[:username]
 @password  = options[:password]
 @filter    = options[:filter] || ['NEW']
 @port      = options[:port] || 143

Good point, I think he might have copied an older version of lib/mailman/receiver/pop3.rb. Will do that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/mailman/receiver/imap.rb
((30 lines not shown))
+ def connect
+ @connection.login(@username, @password)
+ @connection.examine("INBOX")
+ end
+
+ # Disconnects from the IMAP server.
+ def disconnect
+ @connection.logout
+ @connection.disconnected? ? true : @connection.disconnect rescue nil
+ end
+
+ # Iterates through new messages, passing them to the processor, and
+ # deleting them.
+ def get_messages
+ @connection.search(@filter).each do |message|
+ puts "PROCESSING MESSAGE #{message}"
@titanous Owner

This should use the logger, using sentence case.

Yep, I'll correct this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@titanous titanous commented on the diff
lib/mailman/receiver/imap.rb
((36 lines not shown))
+ def disconnect
+ @connection.logout
+ @connection.disconnected? ? true : @connection.disconnect rescue nil
+ end
+
+ # Iterates through new messages, passing them to the processor, and
+ # deleting them.
+ def get_messages
+ @connection.search(@filter).each do |message|
+ puts "PROCESSING MESSAGE #{message}"
+ body = @connection.fetch(message,"RFC822")[0].attr["RFC822"]
+ @processor.process(body)
+ @connection.store(message,"+FLAGS",[Net::IMAP::DELETED])
+ end
+ # Clears messages that have the Deleted flag set
+ @connection.expunge
@titanous Owner

What do you think about moving messages to a different folder instead of deleting them?

Yep, that's going to be the next feature. I'm looking to replace our existing IMAP processor with Mailman, which currently moves processed emails to a new folder.

Here's what I would like to implement: https://github.com/fatfreecrm/fat_free_crm/blob/master/lib/fat_free_crm/dropbox.rb#L146-164

I'm not sure if / how Mailman handles unmatched emails? We use a discard method, which either deletes the message or moves it to an invalid folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@titanous titanous commented on the diff
lib/mailman/receiver/imap.rb
((23 lines not shown))
+ @password = options[:password] if options.has_key? :password
+ @filter = options[:filter] if options.has_key? :filter
+ @port = options[:port] if options.has_key? :port
+ @connection = Net::IMAP.new(options[:server], @port)
+ end
+
+ # Connects to the IMAP server.
+ def connect
+ @connection.login(@username, @password)
+ @connection.examine("INBOX")
+ end
+
+ # Disconnects from the IMAP server.
+ def disconnect
+ @connection.logout
+ @connection.disconnected? ? true : @connection.disconnect rescue nil
@titanous Owner

How about this?

@connection.disconnect unless @connection.disconnected?

Just copying the specs for POP3, I need to return true for this line https://github.com/fatfreecrm/mailman/blob/imap/spec/mailman/receiver/imap_spec.rb#L22

@connection.disconnect unless @connection.disconnected? will return nil if @connection.disconnected? is true, failing the spec

@titanous Owner

Ah, that's fine then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@titanous
Owner

Looks great, I'll merge it once my comments have been addressed. This will need some documentation at some point as well.

@titanous titanous referenced this pull request
Closed

added in IMAP support #18

@ndbroadbent

Here's those changes. I removed that PROCESSING MESSAGE line, since the POP3 receiver doesn't log anything in #get_messages

@titanous titanous merged commit e3c32b3 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
1  .gitignore
@@ -20,6 +20,7 @@ doc
pkg
*.rbc
.bundle
+Gemfile.lock
## PROJECT::SPECIFIC
spec/test-maildir
View
77 lib/mailman/application.rb
@@ -24,6 +24,10 @@ def initialize(&block)
instance_eval(&block)
end
+ def polling?
+ Mailman.config.poll_interval > 0 && !@polling_interrupt
+ end
+
# Sets the block to run if no routes match a message.
def default(&block)
@router.default_block = block
@@ -39,39 +43,33 @@ def run
require rails_env
end
- if !Mailman.config.ignore_stdin && $stdin.fcntl(Fcntl::F_GETFL, 0) == 0 # we have stdin
+ if Mailman.config.graceful_death
+ # When user presses CTRL-C, finish processing current message before exiting
+ Signal.trap("INT") { @polling_interrupt = true }
+ end
+
+ # STDIN
+ # ---------------------------------------------------------------------
+ if !Mailman.config.ignore_stdin && $stdin.fcntl(Fcntl::F_GETFL, 0) == 0
Mailman.logger.debug "Processing message from STDIN."
@processor.process($stdin.read)
+
+ # IMAP
+ # ---------------------------------------------------------------------
+ elsif Mailman.config.imap
+ options = {:processor => @processor}.merge(Mailman.config.imap)
+ Mailman.logger.info "IMAP receiver enabled (#{options[:username]}@#{options[:server]})."
+ polling_loop Receiver::IMAP.new(options)
+
+ # POP3
+ # ---------------------------------------------------------------------
elsif Mailman.config.pop3
options = {:processor => @processor}.merge(Mailman.config.pop3)
Mailman.logger.info "POP3 receiver enabled (#{options[:username]}@#{options[:server]})."
- if Mailman.config.poll_interval > 0 # we should poll
- polling = true
- Mailman.logger.info "Polling enabled. Checking every #{Mailman.config.poll_interval} seconds."
- else
- polling = false
- Mailman.logger.info 'Polling disabled. Checking for messages once.'
- end
-
- connection = Receiver::POP3.new(options)
-
- if Mailman.config.graceful_death
- Signal.trap("INT") {polling = false}
- end
-
- loop do
- begin
- connection.connect
- connection.get_messages
- connection.disconnect
- rescue SystemCallError => e
- Mailman.logger.error e.message
- end
-
- break if !polling
- sleep Mailman.config.poll_interval
- end
+ polling_loop Receiver::POP3.new(options)
+ # Maildir
+ # ---------------------------------------------------------------------
elsif Mailman.config.maildir
require 'maildir'
require 'fssm'
@@ -99,5 +97,30 @@ def process_maildir
end
end
+ private
+
+ # Run the polling loop for the email inbox connection
+ def polling_loop(connection)
+ if polling?
+ polling_msg = "Polling enabled. Checking every #{Mailman.config.poll_interval} seconds."
+ else
+ polling_msg = "Polling disabled. Checking for messages once."
+ end
+ Mailman.logger.info(polling_msg)
+
+ loop do
+ begin
+ connection.connect
+ connection.get_messages
+ connection.disconnect
+ rescue SystemCallError => e
+ Mailman.logger.error e.message
+ end
+
+ break unless polling?
+ sleep Mailman.config.poll_interval
+ end
+ end
+
end
end
View
6 lib/mailman/configuration.rb
@@ -5,7 +5,7 @@ class Configuration
attr_accessor :logger
# @return [Hash] the configuration hash for POP3
- attr_accessor :pop3
+ attr_accessor :pop3, :imap
# @return [Fixnum] the poll interval for POP3 or IMAP. Setting this to 0
# disables polling
@@ -18,12 +18,12 @@ class Configuration
# rails environment loading
attr_accessor :rails_root
- # @return [boolean] whether or not to ignore stdin. Setting this to true
+ # @return [boolean] whether or not to ignore stdin. Setting this to true
# stops Mailman from entering stdin processing mode.
attr_accessor :ignore_stdin
# @return [boolean] catch SIGINT and allow current iteration to finish
- # rather than dropping dead immediately. Currently only works with POP3
+ # rather than dropping dead immediately. Currently only works with POP3
# connections.
attr_accessor :graceful_death
View
3  lib/mailman/receiver.rb
@@ -2,6 +2,7 @@ module Mailman
module Receiver
autoload :POP3, 'mailman/receiver/pop3'
+ autoload :IMAP, 'mailman/receiver/imap'
end
-end
+end
View
54 lib/mailman/receiver/imap.rb
@@ -0,0 +1,54 @@
+require 'net/imap'
+
+module Mailman
+ module Receiver
+ # Receives messages using IMAP, and passes them to a {MessageProcessor}.
+ class IMAP
+
+ # @return [Net::IMAP] the IMAP connection
+ attr_reader :connection
+
+ # @param [Hash] options the receiver options
+ # @option options [MessageProcessor] :processor the processor to pass new
+ # messages to
+ # @option options [String] :server the server to connect to
+ # @option options [Integer] :port the port to connect to
+ # @option options [String] :username the username to authenticate with
+ # @option options [String] :password the password to authenticate with
+ def initialize(options)
+ @processor = options[:processor]
+ @username = options[:username]
+ @password = options[:password]
+ @filter = options[:filter] || ['NEW']
+ @port = options[:port] || 143
+
+ @connection = Net::IMAP.new(options[:server], @port)
+ end
+
+ # Connects to the IMAP server.
+ def connect
+ @connection.login(@username, @password)
+ @connection.examine("INBOX")
+ end
+
+ # Disconnects from the IMAP server.
+ def disconnect
+ @connection.logout
+ @connection.disconnected? ? true : @connection.disconnect rescue nil
@titanous Owner

How about this?

@connection.disconnect unless @connection.disconnected?

Just copying the specs for POP3, I need to return true for this line https://github.com/fatfreecrm/mailman/blob/imap/spec/mailman/receiver/imap_spec.rb#L22

@connection.disconnect unless @connection.disconnected? will return nil if @connection.disconnected? is true, failing the spec

@titanous Owner

Ah, that's fine then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ end
+
+ # Iterates through new messages, passing them to the processor, and
+ # deleting them.
+ def get_messages
+ @connection.search(@filter).each do |message|
+ body = @connection.fetch(message,"RFC822")[0].attr["RFC822"]
+ @processor.process(body)
+ @connection.store(message,"+FLAGS",[Net::IMAP::DELETED])
+ end
+ # Clears messages that have the Deleted flag set
+ @connection.expunge
@titanous Owner

What do you think about moving messages to a different folder instead of deleting them?

Yep, that's going to be the next feature. I'm looking to replace our existing IMAP processor with Mailman, which currently moves processed emails to a new folder.

Here's what I would like to implement: https://github.com/fatfreecrm/fat_free_crm/blob/master/lib/fat_free_crm/dropbox.rb#L146-164

I'm not sure if / how Mailman handles unmatched emails? We use a discard method, which either deletes the message or moves it to an invalid folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ end
+
+ end
+ end
+end
View
17 spec/functional/application_spec.rb
@@ -122,6 +122,23 @@ def send_example
@app.router.instance_variable_get('@count').should == nil
end
+ it 'should poll an IMAP server, and process messsages' do
+ config.imap = { :server => 'example.com',
+ :username => 'chunky',
+ :password => 'bacon' }
+ config.poll_interval = 0 # just poll once
+
+ mailman_app {
+ from 'chunky@bacon.com' do
+ @count ||= 0
+ @count += 1
+ end
+ }
+
+ @app.run
+ @app.router.instance_variable_get('@count').should == 2
+ end
+
it 'should watch a maildir folder for messages' do
setup_maildir # creates the maildir with a queued message
View
44 spec/mailman/receiver/imap_spec.rb
@@ -0,0 +1,44 @@
+require File.expand_path(File.join(File.dirname(__FILE__), '..', '..', '/spec_helper'))
+
+describe Mailman::Receiver::IMAP do
+
+ before do
+ @processor = mock('Message Processor', :process => true)
+ @receiver_options = { :username => 'user',
+ :password => 'pass',
+ :server => 'example.com',
+ :processor => @processor }
+ @receiver = Mailman::Receiver::IMAP.new(@receiver_options)
+ end
+
+ describe 'connection' do
+
+ it 'should connect to a IMAP server' do
+ @receiver.connect.should be_true
+ end
+
+ it 'should disconnect from a IMAP server' do
+ @receiver.connect
+ @receiver.disconnect.should be_true
+ end
+
+ end
+
+ describe 'message reception' do
+ before do
+ @receiver.connect
+ end
+
+ it 'should get messages and process them' do
+ @processor.should_receive(:process).twice.with(/test/)
+ @receiver.get_messages
+ end
+
+ it 'should delete the messages after processing' do
+ @receiver.get_messages
+ @receiver.connection.search(:all).should be_empty
+ end
+
+ end
+
+end
View
26 spec/spec_helper.rb
@@ -3,9 +3,13 @@
require 'fileutils'
require 'mailman'
require 'rspec'
-require 'pop3_mock'
require 'maildir'
+# Require all files in spec/support (Mocks, helpers, etc.)
+Dir[File.join(File.dirname(__FILE__), "support", "**", "*.rb")].each do |f|
+ require File.expand_path(f)
+end
+
unless defined?(SPEC_ROOT)
SPEC_ROOT = File.join(File.dirname(__FILE__))
end
@@ -61,23 +65,3 @@ def setup_maildir
end
end
-
-class FakeSTDIN
-
- attr_accessor :string
-
- def initialize(string=nil)
- @string = string
- end
-
- def fcntl(*args)
- @string ? 0 : 2
- end
-
- def read
- @string
- end
-
-end
-
-$stdin = FakeSTDIN.new
View
85 spec/support/imap_mock.rb
@@ -0,0 +1,85 @@
+# From https://github.com/mikel/mail/blob/master/spec/spec_helper.rb#L192
+
+class MockIMAPFetchData
+ attr_reader :attr, :number
+
+ def initialize(rfc822, number)
+ @attr = { 'RFC822' => rfc822 }
+ @number = number
+ end
+end
+
+class MockIMAP
+ @@connection = false
+ @@mailbox = nil
+ @@marked_for_deletion = []
+
+ def self.examples
+ @@examples
+ end
+
+ def initialize
+ @@examples = []
+ 2.times do |i|
+ @@examples << MockIMAPFetchData.new("To: test@example.com\r\nFrom: chunky@bacon.com\r\nSubject: Hello!\r\n\r\nemail message\r\ntest#{i.to_s}", i)
+ end
+ end
+
+ def login(user, password)
+ @@connection = true
+ end
+
+ def disconnect
+ @@connection = false
+ end
+
+ def logout
+ @@connection = false
+ end
+
+ def select(mailbox)
+ @@mailbox = mailbox
+ end
+
+ def examine(mailbox)
+ select(mailbox)
+ end
+
+ def uid_search(keys, charset=nil)
+ [*(0..@@examples.size - 1)]
+ end
+ alias :search :uid_search
+
+ def uid_fetch(set, attr)
+ [@@examples[set]]
+ end
+ alias :fetch :uid_fetch
+
+ def uid_store(set, attr, flags)
+ if attr == "+FLAGS" && flags.include?(Net::IMAP::DELETED)
+ @@marked_for_deletion << set
+ end
+ end
+ alias :store :uid_store
+
+
+ def expunge
+ @@marked_for_deletion.reverse.each do |i| # start with highest index first
+ @@examples.delete_at(i)
+ end
+ @@marked_for_deletion = []
+ end
+
+ def self.mailbox; @@mailbox end # test only
+
+ def self.disconnected?; @@connection == false end
+ def disconnected?; @@connection == false end
+
+end
+
+require 'net/imap'
+class Net::IMAP
+ def self.new(*args)
+ MockIMAP.new
+ end
+end
View
11 spec/pop3_mock.rb → spec/support/pop3_mock.rb
@@ -1,9 +1,10 @@
-# From http://github.com/mikel/mail/blob/master/spec/spec_helper.rb#L89
+# From https://github.com/mikel/mail/blob/master/spec/spec_helper.rb#L103
class MockPopMail
def initialize(rfc2822, number)
@rfc2822 = rfc2822
@number = number
+ @deleted = false
end
def pop
@@ -17,6 +18,14 @@ def number
def to_s
"#{number}: #{pop}"
end
+
+ def delete
+ @deleted = true
+ end
+
+ def deleted?
+ @deleted
+ end
end
class MockPOP3
View
19 spec/support/stdin_mock.rb
@@ -0,0 +1,19 @@
+class MockSTDIN
+
+ attr_accessor :string
+
+ def initialize(string=nil)
+ @string = string
+ end
+
+ def fcntl(*args)
+ @string ? 0 : 2
+ end
+
+ def read
+ @string
+ end
+
+end
+
+$stdin = MockSTDIN.new
Something went wrong with that request. Please try again.