Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 240 lines (169 sloc) 9.754 kb
9abdae12 » dave
2010-01-17 Initial commit to enigmamachine.
1 = enigmamachine
2
89b550c4 » dave
2010-07-12 Updated the HTTP basic auth part of the README so it's correct.
3 Enigmamachine is a RESTful web-based video processor which queues and encodes
4 videos according to target profiles that you define. Videos must be on a
5 locally mounted filesystem. The processor takes the path to the video, and
6 executes one or more ffmpeg commands on the video. There is a handy web
7 interface for defining encoding tasks, and a restful web service which takes
8 encoding commands.
f534fd29 » dave@boomer
2010-01-18 A little status info.
9
10 Enigmamachine is written using Sinatra, Thin, and Eventmachine.
11
24110add » dave
2010-07-07 Updated the README to reflect current project status.
12 == Why would you want one?
f534fd29 » dave@boomer
2010-01-18 A little status info.
13
24110add » dave
2010-07-07 Updated the README to reflect current project status.
14 If you're not running a server, you probably don't need enigmamachine, because
15 there are already lots of great client-side batch processors for all kinds of
16 different operating systems. However, if you are running a web application,
17 and you want to process uploaded video, enigmamachine offers you a convenient
18 fire-and-forget video encoder.
19
20 The main problem with encoding video on a server is that it takes a really,
21 really long time - you don't want to do it within the scope of a single HTTP
22 request, because you want the user's browser to return to their control as
23 soon as the upload is finished. If the video took ten minutes to upload,
24 you don't want to keep their browser (and your webapp) busy for a
25 further ten minutes while the encoding happens.
26
27 The right way to deal with the uploaded video is to fire up a new thread which
28 can take over responsibility for encoding the video, while your web app goes
29 on its merry way. Unfortunately, this is difficult in some languages (PHP, for
30 example, doesn't have lightweight threading support), and even in languages
31 with good threading support it still sort of sucks. With Enigmamachine, all you
32 need to do to trigger encoding of a video is shoot off an HTTP request, and
33 everything else is handled for you.
34
7305ae90 » dave
2010-07-20 Formatting.
35
24110add » dave
2010-07-07 Updated the README to reflect current project status.
36 == Using it
37
38 Once you've installed the gem (see below), you can do something like:
39
ff2f07dd » dave
2010-07-07 README formatting.
40 mkdir enigma
41 cd enigma
42 enigmamachine start # -d to daemonize
24110add » dave
2010-07-07 Updated the README to reflect current project status.
43
01ab309d » dave
2010-07-10 Some slightly improved usage docs.
44 Then check it out in your browser, at http://localhost:2002. This web interface
45 exists so that you can configure your enigmamachine easily, and check its status.
24110add » dave
2010-07-07 Updated the README to reflect current project status.
46
01ab309d » dave
2010-07-10 Some slightly improved usage docs.
47 Most of the time, though, your enigmamachine will be accessed not through a
cc997d89 » dave
2010-10-04 typo fix
48 browser, but by your program's code. Let's say you want your website to be able
49 to encode video. You write the code for the video upload, so your web app can
bdf0ee5a » dave
2010-07-10 More docs love.
50 receive the video. When the upload is complete, make an HTTP call something like this in your code:
24110add » dave
2010-07-07 Updated the README to reflect current project status.
51
52 POST http://localhost:2002/videos
53
54 with the params:
55
6b29f9ef » dave
2010-07-21 A bit more documentation on callbacks.
56 "video[file]=/foo/bar/blah.mp4" # the full path to a file on your local filesystem
57 "video[callback_url]=http://example.org/call/back/id" # an optional callback url
89b550c4 » dave
2010-07-12 Updated the HTTP basic auth part of the README so it's correct.
58 "encoder_id=1" # the id of an encoder defined in your enigmamachine database
24110add » dave
2010-07-07 Updated the README to reflect current project status.
59
89b550c4 » dave
2010-07-12 Updated the HTTP basic auth part of the README so it's correct.
60 The enigmamachine will run all encoding tasks on the video, while your web
61 application will be free to continue serving requests happily. If a second video
bdf0ee5a » dave
2010-07-10 More docs love.
62 is uploaded while the first one is still encoding, it will be placed in a queue.
24110add » dave
2010-07-07 Updated the README to reflect current project status.
63 Videos are encoded sequentially as they arrive.
64
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
65 All requests, whether they come from your browser or from your code, are
66 protected by HTTP basic auth. By default, the username is
6713b7ad » dave
2010-07-12 Put in a note about config.yml.
67 <i>admin</i> and the password is <i>admin</i>. You can change the username and
68 password in the config.yml file which will be generated on first use.
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
69
6b29f9ef » dave
2010-07-21 A bit more documentation on callbacks.
70 When a video has finished encoding, the main web application might want to know.
068dd6f8 » dave
2010-07-21 Formatting.
71 The optional @video[callback_url]@ parameter tells enigmamachine to execute a GET
6b29f9ef » dave
2010-07-21 A bit more documentation on callbacks.
72 request to the callback url when video encoding is complete.
73
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
74 Programmatic requests in Ruby might look something like this:
75
697f66d8 » dave
2010-07-12 An actually working enigma_client.rb.
76 # enigma_client.rb:
77
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
78 require 'rubygems'
91f82939 » dave
2010-07-12 The damned client really works now. Really!
79 require 'httparty'
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
80
81 class EnigmaClient
82
91f82939 » dave
2010-07-12 The damned client really works now. Really!
83 include HTTParty
84
85 base_uri "localhost:2002"
86
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
87 def initialize (u, p)
91f82939 » dave
2010-07-12 The damned client really works now. Really!
88 @auth = encode_credentials(u, p)
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
89 end
90
f8cc3941 » dave
2010-07-20 Added a callback_url to the video.
91 def post(path_to_video, encoder_id, callback_url)
91f82939 » dave
2010-07-12 The damned client really works now. Really!
92 self.class.post("/videos", {
f8cc3941 » dave
2010-07-20 Added a callback_url to the video.
93 :body => {:video => {
94 :file => path_to_video,
95 :callback_url => callback_url
96 }, :encoder_id => encoder_id},
91f82939 » dave
2010-07-12 The damned client really works now. Really!
97 :basic_auth => @auth})
98 end
99
100 private
101
102 def encode_credentials(username, password)
103 {:username => username, :password => password}
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
104 end
105
106 end
107
91f82939 » dave
2010-07-12 The damned client really works now. Really!
108
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
109 # Let's use it!
110 #
697f66d8 » dave
2010-07-12 An actually working enigma_client.rb.
111 irb
112 require 'enigma_client'
f8cc3941 » dave
2010-07-20 Added a callback_url to the video.
113 EnigmaClient.new("admin", "admin").post("/path/to/your/uploaded/video.mp4", 1, "http://example.org/call/back/id")
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
114
115
313bf0eb » dave
2010-07-12 Added a working wget example to the invocation docs.
116 A simpler, wget-based approach might look like this:
117
f8cc3941 » dave
2010-07-20 Added a callback_url to the video.
118 wget http://admin:admin@localhost:2002/videos --post-data 'video[file]=/path/to/your/video.mp4&video[callback_url]=http://example.org/call/back/id&encoder_id=1'
313bf0eb » dave
2010-07-12 Added a working wget example to the invocation docs.
119
120 <b>Don't use wget like this on a shared host, it's insecure</b> (read the wget
121 docs for more on this). This wget example is just to show you what's going on
122 when using a simple tool that most programmers know how to use.
123
24110add » dave
2010-07-07 Updated the README to reflect current project status.
124 == Encoders and Encoding Tasks
125
19dfbf35 » dave
2010-07-07 Some docs on the whole Encoders and Encoding Tasks thing.
126 When you POST the location of a video to your enigmamachine, you need to tell
127 your EM what you want to do to the video. For example, you might want to take
128 an uploaded video and encode it as a 320x240 FLV video, with a 320x240 JPEG
129 thumbnail, and a 160x120 miniature thumbnail.
130
131 To do this, you'd fire up your enigmamachine by typing
132
133 enigmamachine start
134
135 and go to http://localhost:2002/encoders. Clicking the "new encoder" link will
136 allow you to define a new encoder. Let's call it "Flash video with a couple
137 thumbnails", and save it.
138
139 So, now there's an encoder, but it won't do anything. Let's add some tasks,
140 by clicking on the "new task" link.
141
e0692370 » dave
2010-07-10 README formatting fixes.
142 Give the first task a name, like <i>Encode a 320x240 flv at 25 frames per second</i>.
19dfbf35 » dave
2010-07-07 Some docs on the whole Encoders and Encoding Tasks thing.
143
144 The output file suffix in this case will be *.flv*.
145
146 The encoding command you'll want to use would be
e0692370 » dave
2010-07-10 README formatting fixes.
147 <i>-ab 128 -ar 22050 -b 500000 -r 25 -s 320x240</i>. This cryptic command string
19dfbf35 » dave
2010-07-07 Some docs on the whole Encoders and Encoding Tasks thing.
148 tells ffmpeg to encode a Flash video at 320x240 pixels, with an audio bitrate
149 of 128kbps, an audio sample rate of 22.050khz, and a frame rate of 25fps. You
150 can find out what all the ffmpeg command line switches do by RTFMing at
151 http://www.ffmpeg.org/ffmpeg-doc.html
152
153 You would go on to define a couple more encoding tasks for your encoder.
154 Grabbing a screen frame in ffmpeg can be done with the following command-line
155 switches, which you can put into a task called, let's say,
e0692370 » dave
2010-07-10 README formatting fixes.
156 <i>Grab a 320x240 JPEG thumbnail</i>:
19dfbf35 » dave
2010-07-07 Some docs on the whole Encoders and Encoding Tasks thing.
157
158 -ss 00:00:05 -t 00:00:01 -vcodec mjpeg -vframes 1 -an -f rawvideo -s 320x240
159
160 Rinse and repeat for the 160x120 thumbnail.
161
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
162 == Security considerations
24110add » dave
2010-07-07 Updated the README to reflect current project status.
163
a00ed9be » dave
2010-07-08 Removed security warnings, the app now listens on loopback only.
164 Enigmamachine is set to bind by default to 127.0.0.1 (your system's loopback)
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
165 interface rather than on all network interfaces, so it won't be accessible from
166 other machines.
414ce718 » dave
2010-07-07 Further security warning.
167
168 Making an enigmamachine available on an untrusted network (like
4aba7317 » dave
2010-07-07 A note on security.
169 the Internet) would be a suicidal move on your part, since the code used to
0a63378d » dave
2010-07-09 Some updated security docs.
170 talk to ffmpeg is a simple backtick exec call and you'll be inviting everyone in
171 the world to execute commands on your server, with your effective user
172 permissions.
173
174 When the enigmamachine starts for the first time in a given directory, it will
175 spit out a config.yml file containing a username and password. All requests
176 will need to submit this auth information. This should make enigmamachine
21ee22e1 » dave
2010-07-09 Updated usage docs with a sample ruby client.
177 reasonably safe to use on shared hosts, just make sure nobody can read the
178 config file except the user executing the enigmamachine process.
4aba7317 » dave
2010-07-07 A note on security.
179
180 If you don't know what any of this means, don't run it. I'm not responsible if
181 your enigmamachine screws up your system, allows people to exploit you, or
a00ed9be » dave
2010-07-08 Removed security warnings, the app now listens on loopback only.
182 eats your mother.
24110add » dave
2010-07-07 Updated the README to reflect current project status.
183
184 == Installation
185
186 Enigmamachine is written in Ruby and uses the Sinatra web framework, the
5bf8bdb8 » dave
2010-07-09 Minor docs changes, let's not forget about Thin.
187 Datamapper ORM library, the Eventmachine event-processing library, and the Thin
188 webserver. If you're not a Ruby person, you don't need to care about any of this.
24110add » dave
2010-07-07 Updated the README to reflect current project status.
189 Enigmamachine can be triggered by any language that can send a POST request -
190 so all you PHPistas, python-lovers, droopy-moustachists, or blue-suited
191 java types can all use it just as easy as us fashionable-haircut-language
192 people.
193
194 You can install it as a gem by doing:
195
196 gem install enigmamachine
197
198 If this command doesn't make any sense to you, it's because you don't know that
199 "gems" are ruby code packages, somewhat like apt-get except for ruby code only.
89fc9081 » dave
2010-07-07 Added a security warning and improved install docs.
200 You can install rubygems, necessary sqlite headers, and a C compiler on
201 righteous operating systems by typing:
24110add » dave
2010-07-07 Updated the README to reflect current project status.
202
f96f69e5 » dave
2010-07-10 Added ffmpeg to the list of packages which need to be installed.
203 apt-get install rubygems ruby1.8-dev libopenssl-ruby build-essential libsqlite3-dev ffmpeg # as root
24110add » dave
2010-07-07 Updated the README to reflect current project status.
204
9eb8658c » dave
2010-07-08 Added PATH export stuff for .bashrc into place.
205 You'll need to add the following line to your ~/.bashrc file, as well:
206
207 export PATH=/var/lib/gems/1.8/bin:$PATH
208
5bf8bdb8 » dave
2010-07-09 Minor docs changes, let's not forget about Thin.
209 Then <i>gem install enigmamachine</i> should theoretically work. You'll also need a copy of
469b5d51 » dave
2010-07-07 More docs love.
210 ffmpeg installed and available in your path. On Mac OS X, rubygems should
211 already be installed, although you'll need to have a C compiler available
212 (make sure you install the developer tools that came with your operating
213 system).
24110add » dave
2010-07-07 Updated the README to reflect current project status.
214
215 == Status
216
810a1614 » dave
2010-10-04 Modified the status part of the docs.
217 Enigmamachine should be considered beta quality code. At the same time,
218 crashes and bugs have been infrequent for us, and it's a simple system which
219 is working well. Please let us know of any bugs by filing an issue on the
220 GitHub tracker.
f534fd29 » dave@boomer
2010-01-18 A little status info.
221
9abdae12 » dave
2010-01-17 Initial commit to enigmamachine.
222
223 == Note on Patches/Pull Requests
15101c33 » dave
2010-01-17 Initial commit.
224
9abdae12 » dave
2010-01-17 Initial commit to enigmamachine.
225 * Fork the project.
226 * Make your feature addition or bug fix.
227 * Add tests for it. This is important so I don't break it in a
228 future version unintentionally.
15101c33 » dave
2010-01-17 Initial commit.
229 * Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
9abdae12 » dave
2010-01-17 Initial commit to enigmamachine.
230 * Send me a pull request. Bonus points for topic branches.
231
232 == Copyright
233
15101c33 » dave
2010-01-17 Initial commit.
234 Copyright (c) 2010 Dave Hrycyszyn. See LICENSE for details.
235
f1ea6dc5 » dave
2010-07-22 Adding headlabs credit.
236 Enigmamachine is a headlabs project. See http://labs.headlondon.com for more.
237
3562a03c » dave
2010-09-15 Added a contributors heading to the README.
238 == Contributors
239
240 Dave Hrycyszyn, Dmitry Brazhkin
241
Something went wrong with that request. Please try again.