Skip to content
Newer
Older
100644 106 lines (75 sloc) 4.35 KB
bb0516b initial commit
Daniel Tenner authored
1 h1. What is Num-fu?
2
651098e added upload extractor and updated docs
Daniel Tenner authored
3 Num-fu is a plugin built for "Woobius":http://www.woobius.com, which handles uploads coming via "nginx's upload module":http://www.grid.net.ru/nginx/upload.en.html. More info on that upload module "here":http://brainspl.at/articles/2008/07/20/nginx-upload-module and "here":http://www.motionstandingstill.com/nginx-upload-awesomeness/2008-08-13/.
bb0516b initial commit
Daniel Tenner authored
4
daf9577 Collected some bits and bobs out of our hacked version of AttachmentF…
Daniel Tenner authored
5 It's worth noting that this is not a straightforward adaptation of AttachmentFu to the Nginx Upload Module. I've picked out the functionality that we are actually using, and much of AttachmentFu has been left aside. This plugin has perhaps 5% of the functionality of Attachment-Fu, and no intention to build in things like image processing, S3 support, thumbnails, etc.
6
bb0516b initial commit
Daniel Tenner authored
7 h1. Can I use this?
8
9 Yes, go for it. You may fork it and/or use and abuse it in any way you like.
10
11 h1. How can I use this?
12
daf9577 Collected some bits and bobs out of our hacked version of AttachmentF…
Daniel Tenner authored
13 h2. Usage:
14
651098e added upload extractor and updated docs
Daniel Tenner authored
15 Your model should look something like this:
daf9577 Collected some bits and bobs out of our hacked version of AttachmentF…
Daniel Tenner authored
16
17 <pre>
18 class FileReference < ActiveRecord::Base
19 has_attachment :max_size => 2.gigabytes,
20 :path_prefix => STORAGE_FOLDER
21 end
22 </pre>
23
e2eb227 Doc fix
Daniel Tenner authored
24 In your controller, add an <code>extract_upload</code> call before processing the upload, to ensure that your code works whether or not nginx is active:
651098e added upload extractor and updated docs
Daniel Tenner authored
25
26 <pre>
e2eb227 Doc fix
Daniel Tenner authored
27 class MyController < ApplicationController
28 include NumFu::UploadExtractor
29
30 def upload
31 extract_upload :file_reference
32 FileReference.create(params[:file_reference])
33 end
34 end
651098e added upload extractor and updated docs
Daniel Tenner authored
35 </pre>
36
37 And add the following to your nginx config:
38
39 <pre>
32ec0e1 Updated docs
Daniel Tenner authored
40 location /upload {
41
42 client_max_body_size 2048m;
43
651098e added upload extractor and updated docs
Daniel Tenner authored
44 # Pass altered request body to this location
32ec0e1 Updated docs
Daniel Tenner authored
45 upload_pass /process_upload;
651098e added upload extractor and updated docs
Daniel Tenner authored
46
47 # Store files to this directory
32ec0e1 Updated docs
Daniel Tenner authored
48 upload_store /tmp/uploads;
651098e added upload extractor and updated docs
Daniel Tenner authored
49
50 # Allow uploaded files to be read by everyone
51 upload_store_access user:rw group:rw all:rw;
52
53 # Set specified fields in request body
54 # this puts the orginal filename, new path+filename and content type in the requests params
55 upload_set_form_field $upload_field_name.name "$upload_file_name";
56 upload_set_form_field $upload_field_name.content_type "$upload_content_type";
57 upload_set_form_field $upload_field_name.path "$upload_tmp_path";
58
59 upload_pass_form_field "^_session_id$";
0e62f17 Fixed the upload extractor
Daniel Tenner authored
60 upload_pass_form_field "^folder\[id\]$";
32ec0e1 Updated docs
Daniel Tenner authored
61
62 break;
651098e added upload extractor and updated docs
Daniel Tenner authored
63 }
64
65 # dummy location that needs to be defined. :-(
32ec0e1 Updated docs
Daniel Tenner authored
66 location /process_upload {
67 client_max_body_size 2048m;
68 proxy_pass http://upstream_server;
69 break;
651098e added upload extractor and updated docs
Daniel Tenner authored
70 }
71 </pre>
72
0e62f17 Fixed the upload extractor
Daniel Tenner authored
73 Worth noting:
651098e added upload extractor and updated docs
Daniel Tenner authored
74
75 * You have to explicitly tell nginx what fields to pass, if there are additional fields, on top of the file itself (and there likely will be things like session id)
76 * The location that you post to has to be of a direct route, off root - not /controller/action
0e62f17 Fixed the upload extractor
Daniel Tenner authored
77 * Make sure you include the <code>client_max_body_size</code> and the <code>break</code>s, or you may hit problems
78 * To pass railsy parameters within parameters (e.g. <code>folder[id]</code>), don't forget to escape the <code>[]</code> in the regular expression.
651098e added upload extractor and updated docs
Daniel Tenner authored
79
daf9577 Collected some bits and bobs out of our hacked version of AttachmentF…
Daniel Tenner authored
80 h2. Schema setup
81
82 You will need the following fields set up on your FileReference schema:
83
84 <pre>
85 t.string "filename"
86 t.string "content_type"
87 t.integer "size"
88 t.datetime "created_at"
89 t.datetime "updated_at"
90 t.string "original_filename"
91 t.datetime "deleted_at"
92 </pre>
93
32ec0e1 Updated docs
Daniel Tenner authored
94 h2. How are the files stored?
95
96 Num-Fu stores the files in whatever directory you pass as the <code>path_prefix</code> option. It organises them in subdirectories so that there are no more than 256 files per subdirectory. The folder structure is capable of supporting up to 10000 * 10000 * 256 = 25.6 billion files, via a structure like:
97
98 <pre>
99 0001/0001/c35e6f0358ce7390-IMG_A.JPG
100 0001/0025/d0a706593750e972-IMG_B.JPG
101 0123/0123/dcc3b7d5fc5e146f-IMG_C.JPG
102 </pre>
103
104 The folder structure will start at 0001/0001 and fill out all the way until 9999/9999. I have _not_ tested what happens when you exceed that limit... if you are storing 25 billion files, however, I would suggest you look into writing your own storage handling code (perhaps S3 storage might be more appropriate than filesystem storage).
105
106 The 16-alphanumeric-digit random string prepended to each filename is there to ensure, within reason, that there are no collisions between files with the same name. It is technically possible that two files uploaded with the same filename around the same time will end up in the same directory and overwrite each other. However, based on the usage patterns that we are seeing on Woobius, it's probably more productive to worry about the Earth exploding spontaneously.
Something went wrong with that request. Please try again.