Skip to content
Browse files

Download is hacked but works

  • Loading branch information...
1 parent 1eacbc3 commit 294a9b3fc15a6df441c27a8c7082cb8c3989fb84 Jacob Harris committed
Showing with 213 additions and 12 deletions.
  1. +69 −11 README.rdoc
  2. +2 −1 Rakefile
  3. +132 −0 lib/tweetftp.rb
  4. +10 −0 script/console
View
80 README.rdoc
@@ -1,16 +1,74 @@
= tweetftp
-Description goes here.
-
-== Note on Patches/Pull Requests
-
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
- future version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
- (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
+Finally, a mechanism for sharing files on twitter that works WITHIN twitter
+itself. No more third-party services and cryptic short URLs, now you can share files directly
+with your friends or with the world one tweet at a time.
+
+== Uploading Files
+
+A file is uploaded to the status service as a series of messages:
+
+* One header for the file transfer. This contains the string "==BEGIN", the file name, file permissions, and a modified Base64-encoded string of the file's MD5 checksum with the hashtag "#tweetftp" on the end. We modify the string to remove the trailing = signs and replace the characters '+' and '/' with '-' and '_' respectively. While clients may use the checksum to verify the file was retrieved correctly, this is not a requirement: the checksum is primarily used to associate the status updates with one another. The header is also where clients may add a few hashtags to describe the file.
+* 0 or more Description lines. These include the string ==DESC, the modified checksum string from the header, a numerical index, and freeform text. Clients use the index to stitch together a text description from multiple messages.
+* The file itself. This is transmitted as multiple lines with the following fields: an index (starting from 0), up to 77-bytes of the Base64-uuencoding of the file, and the checksum string.
+* A message footer. This contains the string "==END", the file name again, the total number of file message lines, and the Checksum string again.
+
+Fields for each message type are separated with a single space. In addition,
+if the file transfer is meant to be addressed to a single recipient (rather
+than shared with everybody), each message type is prefixed with the @username
+of the recipient. An example (with the data lines truncated for conciseness)
+is below.
+
+ ==BEGIN random.txt 644 106flmMwmFtfZnkYTf1g-g #tweetftp
+ ==DESC 0 This is a test. This is only a test. But I can write a bunch of words about 106flmMwmFtfZnkYTf1g-g #tweetftp
+ ==DESC 1 it being a test here and see how it works. #foo #bar #baz 106flmMwmFtfZnkYTf1g-g #tweetftp
+ 0 YnJlYWtmYXN0Cmx1bmNoCmRpbm5lcgpzbmFjawplYXRpbmcKZHJpbmtpbmcKZWF0aW5nCm1lYWwK 106flmMwmFtfZnkYTf1g-g #tweetftp
+ 1 CnNhbmR3aWNoCiAgc2FuZHdpY2hlcwogIHNhbW1pY2gKICBiYXAKICBncmluZGVyCiAgaG9hZ2ll 106flmMwmFtfZnkYTf1g-g #tweetftp
+ ...
+ 31 dWVlbg== 106flmMwmFtfZnkYTf1g-g #tweetftp
+ ==END food_terms.txt 32 106flmMwmFtfZnkYTf1g-g #tweetftp
+
+== Downloading Files
+
+To download the files from tweetftp, it is enough to use the MD5
+checksum string to retrieve all the individual fragments and reassemble
+into a single file. This MD5 checksum can either be retrieved directly
+from the message header or footer or could be sent separately to
+intended recipients. Once all of the individual message fragments
+are retrieved, the index fields can be used to reorder the parts into
+a file which is then uudecoded onto local storage. Note that while the
+Base64 checksum should be unique enough to find the file, it is recommended
+that the sender's identity is also factored into the search. Otherwise, it
+might be possible to third parties to corrupt files by sending their own
+spoof messages with the same checksum string.
+
+== Practical Considerations
+
+Security is not specified in this approach. Users have the option of
+using specific security mechanisms to encrypt/sign the file before
+transmitting it. For access control, you can share files on a protected
+account.
+
+Twitter currently allows API clients to only post up to 150 messages per hour.
+This practically limits this mechanism to a maximum speed of 0.019kbps. There
+is hope though! Heavy users of twitter may apply for an elevated rate limit of
+20000 requests per hour, which increases throughput to a comparatively blazing
+2.51kbps. On the bright side, this means that only truly important files will be
+shared via tweetftp, and the journey is more important than reaching the destination,
+isn't it?
+
+Since it would take only 60,000 or so tweets to transmit an MP3, there would
+exist the natural temptation to use this mechanism for illegal file sharing.
+This problem is best addressable through legal and social solutions however,
+and technical approaches are beyond the reach of this document. Remember,
+NEVER use tweetftp to illegally share copyrighted material.
+
+== Roadmap
+
+* Actual tests
+* Checksum verifications of downloads
+* TworrentTracker for tracking public file uploads to twitter
+* Get purchased by Google, Facebook, or a desperate media company.
== Copyright
View
3 Rakefile
@@ -10,7 +10,8 @@ begin
gem.email = "jharris@nytimes.com"
gem.homepage = "http://github.com/harrisj/tweetftp"
gem.authors = ["Jacob Harris"]
- gem.add_development_dependency "thoughtbot-shoulda", ">= 0"
+ gem.add_dependency 'twitter', '>= 0'
+ gem.add_development_dependency "shoulda", ">= 0"
# gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
end
Jeweler::GemcutterTasks.new
View
132 lib/tweetftp.rb
@@ -0,0 +1,132 @@
+require 'rubygems'
+require 'base64'
+require 'digest/md5'
+require 'twitter'
+require 'tempfile'
+
+TWEETFTP_HASHTAG = "#tweetftp"
+
+class Tweetftp
+ def initialize(username, password)
+ auth = Twitter::HTTPAuth.new(username, password)
+ @twitter = Twitter::Base.new(auth)
+ end
+
+ ##
+ # Uploads a file with tweetftp
+ #
+ # @par
+ def upload(file, options={})
+ options[:name] ||= file.gsub(/.+\//, '')
+
+ encoded = `uuencode -m #{file} #{options[:name]}`
+ raise "Error encoding file #{file}" unless $? == 0
+
+ options[:hash] = Base64.encode64(Digest::MD5.digest(encoded)).gsub(/=+$/, '').gsub('+', '-').gsub('/', '_').rstrip
+ count = 0
+
+ encoded.each do |line|
+ line = line.rstrip
+ break if line =~ /^===/
+ if count == 0
+ # info line
+ upload_header_line(line, options)
+ upload_description(options)
+ else
+ upload_data_line(line, count - 1, options)
+ end
+
+ count += 1
+ end
+
+ upload_end_line(count - 1, options)
+ end
+
+ # For the MD5 hash, search for all tweets matching; download, sort by count at beginning
+ def download(hash, sender, options={})
+ options[:save_to] ||= '/'
+ page = 1
+
+ fragments = []
+
+ # hard limit from twitter
+ while page <= 100
+ search = Twitter::Search.new(hash).from(sender.gsub(/^@/, '')).page(page)
+ search = search.to(options[:to].gsub(/^@/, '')) if options[:to]
+
+ count = 0
+
+ search.each do |result|
+ count += 1
+ fragments << result[:text].gsub(/^@[\w_]+\s/, '')
+ end
+
+ break if count == 0 || fragments.any? {|f| f =~ /^==BEGIN/ }
+ end
+
+ # we have all the fragments
+ tf = Tempfile.new 'tweetftp'
+
+ sorted_fragments = fragments.reject {|f| f =~ /^==/}.sort_by {|a| a.to_i }
+
+ puts sorted_fragments.inspect
+
+ mode = 644 # FIXME
+ tf.write "begin-base64 #{mode} tweetftp.tmp\n"
+
+ sorted_fragments.each do |f|
+ if f =~ /(\d+) ([a-zA-Z0-9\+\=]+) ([a-zA-Z0-9\-\_]+) #tweetftp/
+ tf.write $2+"\n"
+ end
+ end
+
+ tf.write "====\n"
+ tf.close
+
+ system("uudecode -o /tweetftp.dat #{tf.path}")
+ end
+
+private
+ def upload_tweet(line, options)
+ status = options.key?(:to) ? "#{options[:to]} " : ''
+ status << line
+
+ #puts status
+ @twitter.update(status)
+ end
+
+ def upload_header_line(line, options)
+ if line =~ /^begin-base64\s(\d+)\s/
+ mode = $1
+ else
+ mode = 600
+ end
+
+ upload_tweet("==BEGIN #{options[:name]} #{mode} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+ end
+
+ def upload_description(options)
+ txt = ''
+ txt += options[:description]
+ txt += ' '
+
+ if options[:keywords]
+ txt += options[:keywords].map {|k| "##{k.gsub(/^#/, '')}"}.join(' ')
+ end
+
+ txt.strip!
+ return if txt.empty?
+
+ txt.gsub(/(.{1,77})( +|$\n?)|(.{1,77})/, "\\1\\3\n").lines.each_with_index do |l,i|
+ upload_tweet("==DESC #{i} #{l.chomp} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+ end
+ end
+
+ def upload_data_line(line, count, options)
+ upload_tweet("#{count} #{line} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+ end
+
+ def upload_end_line(count, options)
+ upload_tweet("==END #{options[:name]} #{count} #{options[:hash]} #{TWEETFTP_HASHTAG}", options)
+ end
+end
View
10 script/console
@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# File: script/console
+irb = RUBY_PLATFORM =~ /(:?mswin|mingw)/ ? 'irb.bat' : 'irb'
+
+libs = " -r irb/completion"
+# Perhaps use a console_lib to store any extra methods I may want available in the cosole
+# libs << " -r #{File.dirname(__FILE__) + '/../lib/console_lib/console_logger.rb'}"
+libs << " -r #{File.dirname(__FILE__) + '/../lib/tweetftp.rb'}"
+puts "Loading tweetftp gem"
+exec "#{irb} #{libs} --simple-prompt"

0 comments on commit 294a9b3

Please sign in to comment.
Something went wrong with that request. Please try again.