Download is hacked but works

README.rdoc

@@ -1,16 +1,74 @@
 = tweetftp
 
-Description goes here.
-
-== Note on Patches/Pull Requests
-
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
-  future version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
-  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
+Finally, a mechanism for sharing files on twitter that works WITHIN twitter
+itself. No more third-party services and cryptic short URLs, now you can share files directly
+with your friends or with the world one tweet at a time.
+
+== Uploading Files
+
+A file is uploaded to the status service as a series of messages:
+
+* One header for the file transfer. This contains the string "==BEGIN", the file name, file permissions, and a modified Base64-encoded string of the file's MD5 checksum with the hashtag "#tweetftp" on the end. We modify the string to remove the trailing = signs and replace the characters '+' and '/' with '-' and '_' respectively. While clients may use the checksum to verify the file was retrieved correctly, this is not a requirement: the checksum is primarily used to associate the status updates with one another. The header is also where clients may add a few hashtags to describe the file.
+* 0 or more Description lines. These include the string ==DESC, the modified checksum string from the header, a numerical index, and freeform text. Clients use the index to stitch together a text description from multiple messages.
+* The file itself. This is transmitted as multiple lines with the following fields: an index (starting from 0), up to 77-bytes of the Base64-uuencoding of the file, and the checksum string.
+* A message footer. This contains the string "==END", the file name again, the total number of file message lines, and the Checksum string again.
+
+Fields for each message type are separated with a single space. In addition,
+if the file transfer is meant to be addressed to a single recipient (rather
+than shared with everybody), each message type is prefixed with the @username
+of the recipient. An example (with the data lines truncated for conciseness)
+is below.
+
+  ==BEGIN random.txt 644 106flmMwmFtfZnkYTf1g-g #tweetftp
+  ==DESC 0 This is a test. This is only a test. But I can write a bunch of words about 106flmMwmFtfZnkYTf1g-g #tweetftp
+  ==DESC 1 it being a test here and see how it works. #foo #bar #baz 106flmMwmFtfZnkYTf1g-g #tweetftp
+  0 YnJlYWtmYXN0Cmx1bmNoCmRpbm5lcgpzbmFjawplYXRpbmcKZHJpbmtpbmcKZWF0aW5nCm1lYWwK 106flmMwmFtfZnkYTf1g-g #tweetftp
+  1 CnNhbmR3aWNoCiAgc2FuZHdpY2hlcwogIHNhbW1pY2gKICBiYXAKICBncmluZGVyCiAgaG9hZ2ll 106flmMwmFtfZnkYTf1g-g #tweetftp
+  ...
+  31 dWVlbg== 106flmMwmFtfZnkYTf1g-g #tweetftp
+  ==END food_terms.txt 32 106flmMwmFtfZnkYTf1g-g #tweetftp
+
+== Downloading Files
+
+To download the files from tweetftp, it is enough to use the MD5
+checksum string to retrieve all the individual fragments and reassemble
+into a single file. This MD5 checksum can either be retrieved directly
+from the message header or footer or could be sent separately to 
+intended recipients. Once all of the individual message fragments 
+are retrieved, the index fields can be used to reorder the parts into
+a file which is then uudecoded onto local storage. Note that while the
+Base64 checksum should be unique enough to find the file, it is recommended
+that the sender's identity is also factored into the search. Otherwise, it
+might be possible to third parties to corrupt files by sending their own
+spoof messages with the same checksum string.
+
+== Practical Considerations
+
+Security is not specified in this approach. Users have the option of
+using specific security mechanisms to encrypt/sign the file before
+transmitting it. For access control, you can share files on a protected
+account.
+
+Twitter currently allows API clients to only post up to 150 messages per hour.
+This practically limits this mechanism to a maximum speed of 0.019kbps. There
+is hope though! Heavy users of twitter may apply for an elevated rate limit of
+20000 requests per hour, which increases throughput to a comparatively blazing
+2.51kbps. On the bright side, this means that only truly important files will be
+shared via tweetftp, and the journey is more important than reaching the destination,
+isn't it?
+
+Since it would take only 60,000 or so tweets to transmit an MP3, there would
+exist the natural temptation to use this mechanism for illegal file sharing.
+This problem is best addressable through legal and social solutions however,
+and technical approaches are beyond the reach of this document. Remember,
+NEVER use tweetftp to illegally share copyrighted material.
+
+== Roadmap
+
+* Actual tests
+* Checksum verifications of downloads
+* TworrentTracker for tracking public file uploads to twitter
+* Get purchased by Google, Facebook, or a desperate media company.
 
 == Copyright
 

Rakefile

@@ -10,7 +10,8 @@ begin
     gem.email = "jharris@nytimes.com"
     gem.homepage = "http://github.com/harrisj/tweetftp"
     gem.authors = ["Jacob Harris"]
-    gem.add_development_dependency "thoughtbot-shoulda", ">= 0"
+    gem.add_dependency 'twitter', '>= 0'
+    gem.add_development_dependency "shoulda", ">= 0"
     # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
   end
   Jeweler::GemcutterTasks.new

lib/tweetftp.rb

@@ -0,0 +1,132 @@
+require 'rubygems'
+require 'base64'
+require 'digest/md5'
+require 'twitter'
+require 'tempfile'
+
+TWEETFTP_HASHTAG = "#tweetftp"
+
+class Tweetftp
+  def initialize(username, password)
+    auth = Twitter::HTTPAuth.new(username, password)
+    @twitter = Twitter::Base.new(auth)
+  end
+
+  ##
+  # Uploads a file with tweetftp
+  #
+  # @par
+  def upload(file, options={})
+    options[:name] ||= file.gsub(/.+\//, '')
+
+    encoded = `uuencode -m #{file} #{options[:name]}`
+    raise "Error encoding file #{file}" unless $? == 0
+
+    options[:hash] = Base64.encode64(Digest::MD5.digest(encoded)).gsub(/=+$/, '').gsub('+', '-').gsub('/', '_').rstrip
+    count = 0
+
+    encoded.each do |line|
+      line = line.rstrip
+      break if line =~ /^===/
+      if count == 0
+        # info line
+        upload_header_line(line, options)
+        upload_description(options)
+      else
+        upload_data_line(line, count - 1, options)
+      end
+
+      count += 1
+    end
+
+    upload_end_line(count - 1, options)
+  end
+
+  # For the MD5 hash, search for all tweets matching; download, sort by count at beginning
+  def download(hash, sender, options={})
+    options[:save_to] ||= '/'
+    page = 1
+
+    fragments = []
+
+    # hard limit from twitter
+    while page <= 100
+      search = Twitter::Search.new(hash).from(sender.gsub(/^@/, '')).page(page)
+      search = search.to(options[:to].gsub(/^@/, '')) if options[:to]
+
+      count = 0
+
+      search.each do |result|
+        count += 1
+        fragments << result[:text].gsub(/^@[\w_]+\s/, '')
+      end
+
+      break if count == 0 || fragments.any? {|f| f =~ /^==BEGIN/ }
+    end
+
+    # we have all the fragments
+    tf = Tempfile.new 'tweetftp'
+
+    sorted_fragments = fragments.reject {|f| f =~ /^==/}.sort_by {|a| a.to_i }
+
+    puts sorted_fragments.inspect
+
+    mode = 644 # FIXME
+    tf.write "begin-base64 #{mode} tweetftp.tmp\n"
+
+    sorted_fragments.each do |f|
+      if f =~ /(\d+) ([a-zA-Z0-9\+\=]+) ([a-zA-Z0-9\-\_]+) #tweetftp/
+        tf.write $2+"\n"
+      end
+    end
+
+    tf.write "====\n"
+    tf.close
+
+    system("uudecode -o /tweetftp.dat #{tf.path}")
+  end
+
+private
+  def upload_tweet(line, options)
+    status = options.key?(:to) ? "#{options[:to]} " : ''
+    status << line
+
+    #puts status
+    @twitter.update(status)
+  end
+
+  def upload_header_line(line, options)
+    if line =~ /^begin-base64\s(\d+)\s/
+      mode = $1
+    else
+      mode = 600
+    end
+
+    upload_tweet("==BEGIN #{options[:name]} #{mode} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+  end
+
+  def upload_description(options)
+    txt = ''
+    txt += options[:description]
+    txt += ' '
+
+    if options[:keywords]
+      txt += options[:keywords].map {|k| "##{k.gsub(/^#/, '')}"}.join(' ')
+    end
+
+    txt.strip!
+    return if txt.empty?
+
+    txt.gsub(/(.{1,77})( +|$\n?)|(.{1,77})/, "\\1\\3\n").lines.each_with_index do |l,i|
+      upload_tweet("==DESC #{i} #{l.chomp} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+    end
+  end
+
+  def upload_data_line(line, count, options)
+    upload_tweet("#{count} #{line} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+  end
+
+  def upload_end_line(count, options)
+    upload_tweet("==END #{options[:name]} #{count} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+  end
+end

script/console

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# File: script/console
+irb = RUBY_PLATFORM =~ /(:?mswin|mingw)/ ? 'irb.bat' : 'irb'
+
+libs =  " -r irb/completion"
+# Perhaps use a console_lib to store any extra methods I may want available in the cosole
+# libs << " -r #{File.dirname(__FILE__) + '/../lib/console_lib/console_logger.rb'}"
+libs <<  " -r #{File.dirname(__FILE__) + '/../lib/tweetftp.rb'}"
+puts "Loading tweetftp gem"
+exec "#{irb} #{libs} --simple-prompt"