This project consists of simple Servlet, packaged in a WAR file ready to be deployed in JBoss 7.x AS or Torquebox. The stream is guaranteed to be 1KB wide, so its memory footprint is quite ridiculous. And it's fast. Moreover, no external Java libraries are used.
While all the cool kids in Ruby are busy streaming stuff from the server, there are other equally cool kids who just want to stream stuff to the server. But, is it possible to accomplish this task using only Ruby web frameworks? And more importantly: if Rack is the de-facto engine for writing web frameworks in Ruby, is it enough? The answer is "no it is not", although good progress is being made in finding a decent, stream-friendly successor
Stream-uploading big files with Ruby Rack 1.x is impossible. The Problem is that Rack would read the entire request body into memory, and there are no obvious ways to interact with raw requests and responses in this framework. AFAIK, to accomplish this task, people just use non-Ruby stuff, such as Node.js, JavaEE or even the venerable FastCGI. And if you still don't believe me, just read this.
Under Ruby, instead of monkey-patching Rack, many people use Goliath. Goliath handles streaming,
differently, even though it's Rack-based. But if you stick with your favorite
Rack-based framework, perhaps you could use JRuby and jcommons-rack-upload, which
wraps env['rack.input'] with a rewindable java.io.ByteArrayInputStream. But alas, this later approach is just another monkey-patch.
In my case, the motivation came from Trantor, an internal doc archiving system we are developing. Trantor is distributed,
so the web front-end and API (Sinatra/Rack app) lives in a different server than the file server back-end (another Sinatra/Rack app).
The front-end app also hosts a FileServiceProxy < Sinatra::Base proxy for the GET, DELETE, OPTIONS and HEAD XMLHttpRequest:s
coming from the HTML5 client, which uses jQuery-File-Upload. Because of
the limitations of Rack mentioned above, the POST requests (file creation) must be routed through the Servlet.
Last but not least, I wanted to learn how to use Mockito to test Servlets. I ended up using PowerMockito (PowerMock + Mockito)
because Mockito by itself won't allow mocking final classes, such as Java's URL.
The client can send files in three ways:
- x-www-form-urlencoded (either one or multiple files)
- Single-file binary streams with known or unknown content-length.
Please be aware, that there is Trantor-specific code living under cc.abstra.trantor.wcamp that takes care of:
- Document metadata archiving after a successful POST from the API
- Forwarding the request's OAuth 2.0 credentials to the OmniAuth middleware living in the front-end and processing authorization responses.
Delete this package and the references to its classes from your clone.
Its configuration is done via src/main/webapp/web.xml is simple:
…
<servlet>
…
<init-param>
<param-name>targetUri</param-name>
<param-value>http://remotehost:9090/files</param-value>
</init-param>
</servlet>
…The resulting WAR will be deployed at the /uploader context. If you wish to change it, then edit src/main/webapp/jboss-web.xml.
This is a Maven 3 project. Mosey along: mvn clean package. Please make sure (Open)JDK 1.7 is installed and used (ie. update-alternatives).
After the build process is complete, drop target/stream-uploader.war into $JBOSS_HOME/standalone/deployments or execute
mvn -Plocal-deploy clean package or mvn -Premote-deploy -DremoteIp=a.b.c.d clean package. See pom.xml for more details.
- More tests
- Support chunked file uploads
- Support for XHR2 upload modes
Licensed under the Apache License, Version 2.0. See LICENSE file for more details.
The inspiration came from this StackOverflow post and I stole good implementation
ideas from the HTTP-Proxy-Servlet GitHub project by dsmiley. The HttpHeaders
class comes from the Guava project and MockServletInputStream comes from two-tiers-utils