Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Merge pull request #94 from twinturbo/examples

Add a getting started guide and examples of common tasks
  • Loading branch information...
commit 413c5cc75ff3849293f62b75054342de4abba71b 2 parents 9f8c20d + 5ce6bc0
@wycats wycats authored
View
268 GETTING_STARTED.md
@@ -0,0 +1,268 @@
+# Rake::Pipeline Basics
+
+`Rake::Pipeline` provides a basic extraction over a build process. It
+doesn't have many filters out of the box, but it is very powerful. This
+guide gives you an introduction to `Rake::Pipeline`. `Rake::Pipeline`
+was originally designed for frontend development (HTML, CSS,
+Javascript). The examples assume this type of project just to create
+some context.
+
+## Getting Started
+
+`Rake::Pipeline` comes with two main functionalities out of the box:
+matching and concatentation. You can specify a match (IE: all css files)
+then run them through a concatenation filter (combine them into one
+file). There is also an order concatenation filter which allows you to
+specify order (A should come before B).
+
+Your pipeline is written in an `Assetfile`. The `Assetfile` uses a nice
+DSL to make things easier. The `Assetfile` should live in your project's
+root directory.
+
+Let's get started by writing a basic pipeline. Assume we have this
+directory structure:
+
+```
+/source
+ /css
+ /javascript
+/compiled
+Assetfile
+```
+
+The pipeline should simply concatenate all the individual JSS and CSS
+files into single files. So the JSS and CSS directories are the inputs
+and 2 files are outputs. The `source` directory is input and the output
+will go into `compiled`. Here's the `Assetfile` to do just that:
+
+```ruby
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+
+ # Select all the CSS files
+ match "css/**/*.css" do
+
+ # concatenate all files in directory into a file named
+ # "application.css"
+ concat "application.css"
+ end
+
+ # Repeat for javascript files
+ match "javascript/**/*.js" do
+ concat "application.js"
+ end
+end
+
+# Set the Pipeline's output directory
+output "compiled"
+```
+
+Now run `rakep build` from the project root to compile everything.
+Given there are files in `source/css` and `source/javascript` you will
+see files in `compiled` named `application.js` and `application.css`.
+You've just written your first pipeline!
+
+## Previewing Your Work
+
+`Rake::Pipeline` also comes with a bundled preview server. Let's add an
+HTML file to the source directory to serve the compiled site. Here's the
+HTML:
+
+```html
+<!-- source/index.html -->
+
+<!DOCTYPE html>
+<html>
+<head>
+ <meta charset="utf-8">
+ <title>Rake::Pipeline Example</title>
+ <link rel="stylesheet" href="/application.css">
+</head>
+<body>
+ <script src="/application.js"></script>
+</body>
+</html>
+```
+
+Save that file in `source/index.html`. Now we must add some more code to
+the `Assetfile` to copy the HTML file into the output directory.
+`Rake::Pipeline` also bundles a copy filter. Note that this isn't
+actually a separate filter, but a file that concatenates itself to a
+different location. In short, `concat` is aliased to `copy` as well.
+Here's the updated `Assetfile`:
+
+```ruby
+# input defines operations on a set of files. All files processed in
+# this block go into the output directory
+input "source" do
+
+ # Select all the CSS files
+ match "css/**/*.css" do
+
+ # concatenate all files in directory into a file named
+ # "application.css"
+ concat "application.css"
+ end
+
+ # Repeat for javascript files
+ match "javascript/**/*.js" do
+ concat "application.js"
+ end
+
+ # Explicitly select the HTML file. We don't want to copy
+ # over anything else
+ match "index.html" do
+
+ # copy also accepts a block. When called without any arguments it
+ # simply uses the same filename
+ copy
+ end
+end
+
+# Set the Pipeline's output directory
+output "compiled"
+```
+
+Now we can run `rakep server` inside the projects root. You'll see some
+output in the terminal with a URL to connect to. Now you an preview your
+work as you go.
+
+## Writing Filters
+
+It's very likely that you'll need to do more than just copy and
+concatenate files. You must write your own filters to do this. Luckily,
+writing filters is pretty easy. Filters are classes that have
+`generate_output` method. This is core API requirement. There also other
+method you may implement, but this is the most important. Let's take a
+stab at writing a coffeescript filter.
+
+Filters are Ruby classes. They map a set of inputs to outputs and
+finally generate the output. Here is an absolute bare bones filter:
+
+```ruby
+# Inherit from Rake::Pipeline::Filter to get basic API implemented
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+
+ # This method takes the input files and does whatever is required
+ # to generate the proper output.
+ def generate_output(inputs, output)
+ inputs.each do |input|
+ output.write input.read
+ end
+ end
+end
+```
+
+Notice `generate_output`'s method signature: `inputs` and `output`. The
+default semantic is to take N input files and map them to one output
+file. You can overide this or do much more fancy things. This is covered
+in a different file. We could use this filter like this:
+
+```ruby
+input "source" do
+ match "**/*.coffee" do
+ filter CoffeeScript
+ end
+end
+
+output "compiled"
+```
+
+Now this filter doesn't do anything at the moment besides copy the
+files. Time to implement coffeescript compiling.
+
+```ruby
+require "coffee-script"
+
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+ def generate_output(inputs, output)
+ inputs.each do |input|
+ output.write CoffeeScript.compile(input.read)
+ end
+ end
+end
+```
+
+Great! Now the CoffeeScript files are compiled to JavaScript. However,
+you may have noticed they are compiled "in place". This means
+`source/app.coffee` will become `source/app.coffee` but as JavaScript.
+This works in our simple example, but what happens when we need to work
+with Javascript later in the pipeline or next build steps expect ".js"
+files? The filter has to customize the name. The most correct thing to
+do is make the output file has the same name except as ".js".
+
+This behavior is defined in the filter's initializer. This may seem odd
+to you. It was odd to me until I understood what was happening.
+`Rake::Pipeline` instantiates all the filters in order to setup how
+input files map to output files before the pipeline is compiled. This is
+how one filter can use another's outputs for inputs. This order must be
+known at compile time, so that's why it happens here. The internal API
+expects a block that takes a path and maps it to an output path.
+
+```ruby
+require "coffee-script"
+
+class CoffeeScriptFilter < Rake::Pipeline::Filter
+ def initialize(&block)
+ &block ||= proc { |input| input.path.gsub(/\.coffee, ".js")
+ super(&block)
+ end
+
+ def generate_output(inputs, output)
+ inputs.each do |input|
+ output.write CoffeeScript.compile(input.read)
+ end
+ end
+end
+```
+
+Let's take a fresh look at the `Assetfile` now.
+
+```ruby
+input "source" do
+ match "javascript/**/*.coffee" do
+ # Compile all .coffee files into.js
+ filter CoffeeScript
+ end
+
+ # Select the JS generated by previous filters.
+ match "javascript/**/*.js" do
+ concat "application.js"
+ end
+
+ match "css/**/*.css" do
+ concat "application.css"
+ end
+
+ match "index.html" do
+ copy
+ end
+end
+
+output "compiled"
+```
+
+Calling the filter without a block uses the default block in the filter.
+The default block that replaces ".js" with ".coffee". This is defined
+with `||=` in the initializer. Conversely you could call `filter` with a
+block and do what you want. Here's an example:
+
+```ruby
+# output all coffeescript files as "app.coffee.awesome"
+filter CoffeeScript do |input|
+ "#{input.path}.awesome"
+end
+```
+
+That covers the basics of writing filters. There is much more you can do
+with filters that are outside the scope of this guide. You can find many
+useful (as well as plenty of examples) in the
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+project.
+
+That also concludes this guide. You should know everything you need to
+know to get started writing your own pipelines now. There is still much
+to cover though. You can find additonal information in the `examples`
+directory. If you'd like to add anything to this guide or find an error
+please open a pull request to fix it.
View
7 README.markdown
@@ -2,3 +2,10 @@
The canonical documentation for Rake::Pipeline is hosted at
<a href="http://rubydoc.info/github/livingsocial/rake-pipeline/master/file/README.yard">rubydoc.info</a>.
+
+New users are recommended to read `GETTING_STARTED.md` before anything
+else. Additional examples can be found in the `examples` directory.
+
+Users are also recommended to checkout
+[rake-pipeline-web-filters](https://github.com/wycats/rake-pipeline-web-filters)
+for commonly used filters.
View
12 examples/copying_files.md
@@ -0,0 +1,12 @@
+The `copy` method also accepts an array. You can use this to copy the
+file into multiple locations. Here's an example `Assetfile`:
+
+```
+input "source" do
+ match "*.js" do
+ copy do |input|
+ ["/staging/#{input.path}", "/production/#{input.path}"]
+ end
+ end
+end
+```
View
37 examples/minifying_files.md
@@ -0,0 +1,37 @@
+Minifying files is a very common task for web developers. There are two
+common uses cases:
+
+1. Create an unminified and minified file
+2. Generate a minified file from an unminifed file.
+
+Doing #2 is very easy with rake pipeline. Doing #1 is slightly more
+complicated. For these examples assume there is a `MinifyFilter` that
+actually does the work.
+
+Doing the first use case creates a problem. Filters are destructive.
+That means if you change one file (move it, rewrite) it will not be
+available for the next filters. For example, taking `app.js` and
+minifying it will remove the unminfied source from the pipeline.
+
+You must first duplicate the unminified file then minify it. Here's an
+`Assetfile`
+
+```ruby
+input "source" do
+ match "application.js" do
+ # Duplicate the source file for future filters (application.js)
+ # and provide duplicate in "application.min.js" for minifcation
+ copy ["application.js", "application.min.js"]
+ end
+
+ match "application.min.js" do
+ filter MinifyFilter
+ end
+end
+
+output "compiled"
+```
+
+Now you have two files: `application.js` and `application.min.js`. You
+can use this same technique everytime you need to transform a file and
+keep the source around for future filters.
View
67 examples/modifying_pipelines.md
@@ -0,0 +1,67 @@
+You may want to do some trickery with your input files. Here is a use
+case: you need to sort your files in a custom way. The easiest way to do
+this is to insert a new pipeline into your pipeline. This pipeline must
+act like a filter because it will be used as such. Let's start out by
+describing the most basic pipeline:
+
+```ruby
+class PassThroughPipeline < Rake::Pipeline
+ # this need to act like a filter
+ attr_accessor :pipeline
+
+ # simply return the original input_files
+ def output_files
+ input_files
+ end
+
+ # this is very imporant! define this method
+ # to do nothing and files will not be copied
+ # to the output directory
+ def finalize
+ end
+end
+```
+
+At this point you can insert it into your pipeline:
+
+```ruby
+input "**/*.js" do
+ # stick our pass through
+ pass_through = pipeline.copy PassThroughPipeline
+ pipeline.add_filter pass_through
+
+ # now continue on with your life
+ concat "application.js"
+end
+```
+
+Now we can begin to do all sorts of crazyness in this pass through
+pipeline. You could expand directories to groups of files or you could
+collapse then. You could even skip files if you wanted to. Hell, you can
+even sort them--and that's what we're going to do. So let's get going
+
+```ruby
+class SortedPipeline < PassThroughPipeline
+ def output_files
+ super.sort do |f1, f2|
+ # just an easy example of reverse sorting
+ f2.fullpath <=> f1.fullpath
+ end
+ end
+end
+```
+
+Now add it to the pipeline:
+
+```ruby
+input "**/*.js" do
+ # stick our pass through
+ pass_through = pipeline.copy SortedPipeline
+ pipeline.add_filter pass_through
+
+ # now continue on with your life
+ concat "application.js"
+end
+```
+
+Voila! You can sort stuff. Let your mind run wild with possibilities!
View
77 examples/multiple_pipelines.md
@@ -0,0 +1,77 @@
+Your build process may become to complex to express in terms of a single
+`input` block. You can break your `Assetfile` into multiple input
+blocks. You can also use an `input`'s output as the input for a new
+`input` block.
+
+Say you have different pipelines for different types of files. There is
+one for JS, CSS, and other assets. The output of these 3 different build
+steps needs be the input for the next build step. You can express that
+rather easily with `Rake::Pipeline`. Here are the pipelines for the
+different types. The internals are left out because they are not
+relavant to this example.
+
+```ruby
+# Stage 1
+input "js" do
+ # do JS stuff
+end
+
+input "css" do
+ # do css stuff
+end
+
+input "assets" do
+ # do asset stuff
+end
+```
+
+Now let's add a line at the top of the `Assetfile`. Let's stay that all
+the pipelines should output into one directory.
+
+```ruby
+# All `input` blocks will output to `tmp/stage1` unless output is
+# set again
+output "tmp/stage1"
+
+input "js" do
+ # do JS stuff
+end
+
+input "css" do
+ # do css stuff
+end
+
+input "assets" do
+ # do asset stuff
+end
+```
+
+Now let's hookup stage 2.
+
+```ruby
+# Stage 1
+output "tmp/stage1"
+input "js" do
+ # do JS stuff
+end
+
+input "css" do
+ # do css stuff
+end
+
+input "assets" do
+ # do asset stuff
+end
+
+# Stage 2
+# output of next input block should go to the real output directory
+output "compiled"
+input "tmp/stage1" do
+ # do stage 2 stuff
+end
+```
+
+You can repeat this process over and over again for as many stages as
+you like. Just remember that the final input should output to where you
+want the final files. Also, keep the intermediate build steps inside
+temp directories so they are ignored by source control.
Please sign in to comment.
Something went wrong with that request. Please try again.