-
Notifications
You must be signed in to change notification settings - Fork 0
License
Unknown, GPL-2.0 licenses found
Licenses found
Unknown
LICENSE-BSD.txt
GPL-2.0
LICENSE-GPLv2.txt
reebalazs/gf.rejuice
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
gf.rejuice ========== **`gf.rejuice` provides additional tools for developers to use `Juicer` for the compression of Javascript and CSS resources, in the context of python web applications that run via WSGI.** **Q:** Is this for me, shall I use it? **A:** If you develop javascript with some python based (WSGI) application, and you already use Juicer, or consider using some method for compressing your resources, then you may find `gf.rejuice` useful. **This is a package under active development.** **Q:** Is is safe to use it? **A1:** No. Although it is well tested, it may contain bugs, or may not support your use case. Things can also change substantially, in a following version. **A2:** Yes, it is safe, because you actually only use it for development. It is not even installed on the production website. So, the risk is minimal. Objectives ---------- `gf.rejuice` attempts to provide additional tools for developers to use `Juicer` in the context of python web applications that run via WSGI. `Juicer`_ is a CSS and JavaScript packaging tool, that provides a method to compress (minify) and concatenate the resources for a website or a software package. (Juicer supports the widely used and stable `YUI Compressor`_.) This compression happens manually, offline. Then, if this resource is used in a website page, the page can use the compressed resources directly. .. _Juicer: http://cjohansen.no/en/ruby/juicer_a_css_and_javascript_packaging_tool .. _YUI Compressor: http://developer.yahoo.com/yui/compressor/ It is, however, very difficult or close to impossible to debug and develop websites with fully compressed resources. As a developer, my every day work routine needs an easy way to switch to a "development mode", where I am able to access the original set of uncompressed, undeveloped resources. One possible solution for this would be to provide two sets of resources from the website pages: one for production that contains the compressed resources, and one for development. This can be achieved in several ways, for example, by providing two sets of resources from the html headers conditionally, or use some kind of registry that supports a Development / Production mode switch. The problem with these approaches is that they almost always require changes in the software and possibly extra administration, redundancy, which can be especially painful if the project has many resources. As an alternate solution, to avoid a dual set of resources, some tool can be used that supports the compression and minification of the original sources on the fly. This however has the disadvantage that due to the fact that even the most robust compression methods are prone to errors, it is more difficult to verify the validity of the produced resources, than in offline mode. `gf.rejuice` attempts to provide a simple way to aid this process. In comparision with existing tools, that support the compression and minification of the original sources on the fly, `gf.rejuice` supports a workflow the other way around. It takes the minified resources as the reference, and provides access to the uncompressed resources for development. It does this by providing a way to transparently switch a site that contains and refers the compressed (minified) resources only, into development mode. It does this without the need to change the original software and templates. In addition, it also provides a way to automate the process of compressing or recompressing the resources for production, in case this becomes necessary due to some changes that have been made to the sources. License ------- `gf.rejuice` is dual-licensed by the GPLv2 and BSD licenses. You have to choose any one or both of these licenses for the redistribution of this software. `gf.rejuice` does not contain or redistribute Juicer itself, Juicer still needs to be installed separately, but even this is not necessary for each use case. Installation ------------ You need to install the `gf.rejuice` package to be importable from your project. If you use buildout, this can be done by adding `gf.rejuice` to your egg section:: eggs = .... gf.rejuice In addition, you need to configure a `paster.ini` file that you will use for development. To use the current newest development version, you can download the package from its source repository and install it package manually. Alternately, you can use a recipe like `gf.recipe.bzr` or similar from your buildout to install the software directly from the source repository:: [src] recipe = gf.recipe.bzr urls = bzr+ssh://gfpublic@greenfinity.hu/home/gfpublic/gf.rejuice/trunk gf.rejuice (To use gf.recipe.bzr, you need to have Bazaar installed on your computer.) Simple configuration -------------------- To configure `gf.rejuice`, you need to provide a development configuration for paster. This can be done by copying your original ini file to a second ini file. Then, add the necessary configuration to the development (second) ini. For example, your application had a paster configuration file `myapp.ini`, and you create a second `myapp-devel.ini` from it. Following this, you can use paster with the original, unchanged ini file `myapp.ini` to run the application in production mode, and the `myapp-devel.ini` file will be used to run the application in development mode. The following describes the parts to be added to `myapp-devel.ini` to produce a simple working configuration:: [filter:rejuice] use = egg:gf.rejuice#develjuice url_prefix = /static filepath = egg:my.package/views/static # Resource composition: default.min.js = default.js min/default.min.css = custom/default.css [pipeline:main] pipeline = ... rejuice main_app The WSGI filter section """"""""""""""""""""""" In the above example, the `[filter:rejuice]` section defines the wsgi middleware that does the transformation from production resource urls produced by the the `main_app`, to the corresponding development resource urls. Which application is generating the page to be transformed, or in which particular method this happens, is ambivalent for this process. The `url_prefix` and `filepath` parameters are needed to specify *where* the resources reside, following that a resource list specifies *what* resources the WSGI filter should consider. url_prefix, filepath: specifying the static tree of resources """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""" The `url_prefix` and `filepath` parameters must always stand together with the actual list of resources. Together they specify a static tree in your application. `url_prefix` specify the prefix of the url where the root of this static tree appears in the application's url space. The setting:: url_prefix = /static specifies that the `http://127.0.0.1:6543/static` url is the root of the static tree that your application publishes. `filepath` specifies where the same static tree can be found in the filesystem:: filepath = views/static In this example, the path is constructed relatively to the directory of the `myapp-devel.ini` configuration file that holds the declaration. Note that the `gf.rejuice` middleware does not actually publish this static tree, it still is the responsibility of your application. The parameters are needed for url rewrite. Specifying resource composition """"""""""""""""""""""""""""""" With the rest of the section content, you can specify a list of javascript or css resources here. Anything ending with `.js` or `.css` will be considered to be a resource:: default.min.js = default.js min/default.min.css = custom/default.css Each line contains two values here. The value on the left of the equal sign specifies the compressed, minified resource (the result of compression). The right value specified the original, uncompressed source (the origin of compression). All file paths are applied relatively from the root of the static tree, specified by `url_prefix` and `filepath`. Later we will see that it is also possible to produce one resource from a list of resources, which, in that case will be compressed and concatenated to a single resource. The configuration allows a list of resources in multiple lines:: default.min.js = dependency.js ... default.js Result of the configuration --------------------------- Let's assume that you have the following resources specified from the head section of your html page:: <script src="http://127.0.0.1:6543/static/default.min.js" type="text/javascript"></script> <link href="http://127.0.0.1:6543/static/min/default.min.css" type="text/css" rel="stylesheet"> The middleware will transform this to a website that loads the original sources:: <script src="http://127.0.0.1:6543/static/default.js" type="text/javascript"></script> <link href="http://127.0.0.1:6543/static/custom/default.css" type="text/css" rel="stylesheet"> Juicer also allows to specify dependencies for the resources. The dependent resources may also contain further dependencies. For further explanation, please read the documentation of `juicer`. `default.js`:: /* @depends a/source_a.js @depends b/source_b.js */ `custom/default.css`:: @import ../a/style_a.css @import ../b/style_b.css The middleware will add the dependent javascript resources to the html headers. For css this is not needed, as `@import` describes the inclusion dependencies in a native way for the browser. The end result will be a html that contains the full set of original resources:: <script src="http://127.0.0.1:6543/static/a/source_a.js" type="text/javascript"></script> <script src="http://127.0.0.1:6543/static/b/source_b.js" type="text/javascript"></script> <script src="http://127.0.0.1:6543/static/default.js" type="text/javascript"></script> <link href="http://127.0.0.1:6543/static/custom/default.css" type="text/css" rel="stylesheet"> For more complex examples on how to specify dependencies for `juicer`, you can look at the sources of the `Bottlecap UI`_, that provides the dependencies to compress the `jquery-ui` framework. .. _Bottlecap UI: https://github.com/Pylons/bottlecap Two ways to produce the same result """"""""""""""""""""""""""""""""""" Rather than specifying the dependencies in the style of Juicer, by using `@import` for css and `@depends` for js, the same results can be achieved directly from the configuration as well:: default.min.js = a/source_a.js b/source_b.js default.js min/default.min.css = a/style_a.css b/style_b.css custom/default.css This configuration could replace and is equivalent with the usage of `@import` and `@depends` in the above example. Which one of the two ways are better to use, depends on use case and personal taste. Compression ----------- Although it is quite easy to use Juicer directly from the command line to produce the compressed resources, `gf.rejuice` provides an automation for this. To use Juicer, you must have Juicer and its dependencies already installed. The documentation of Juicer describes this simple process. This installation (including java and ruby) does not need to be present for the middleware to work, only for carrying out the actual compression. If you use buildout, you can enter the following from the command line:: bin/rejuice paster_devel.ini filter:rejuice With the example ini file described earlier, this will produce or reproduce both `default.min.js` and `default.min.css` from their original sources. `filter:rejuice` here refers to the juicer definition section from the ini file. Any extensions referred from this section are also processed. An extension section can also be specified directly:: bin/rejuice another_config_file.ini juice_resources If you do not use buildout, you can also run the `rejuice_script.py` file directly:: python path/to/package/gf/rejuice/rejuice_script.py paster_devel.ini juice_resources Advanced configuration ---------------------- In this section we explain the usage of all parameters. They belong to two groups: the first group specifies which urls the middleware should consider for transformation at all, the second group gives control over selecting the resources that should be considered, by specifying a static tree and by specifying the resource composition itself. Affecting which urls the middleware should consider """"""""""""""""""""""""""""""""""""""""""""""""""" The task of the middleware is to transform only resources that are on the same server as where the page is. So it needs to decide which resources are local and which not. By default, the middleware considers those resources as local, whose domain matches the domain of the currently served page. To do this, nothing needs to be specified. For example, if your application is served from this url:: http://127.0.0.1:6543/ Then the following resources will be considered local, if they appear in the page header:: http://127.0.0.1:6543/default.js http://127.0.0.1:6543/static/default.css /any/absolute/path/my.js any/relative/path/my.css But the following resources will be considered remote, and thus ignored:: http://github.com/jquery/qunit/raw/master/qunit/qunit.js http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.2/themes/smoothness/jquery-ui.css http://localhost:6543/static/default.css This is adequate for most web applications, that generate resource names in a way that they use the base domain of the currently served page. If this is not a case (note that `localhost` would not be accepted as local if the page is served on `127.0.0.1`), or the use case needs finer control, there are two parameters that make this possible. base_urls: specifying the url host to consider '''''''''''''''''''''''''''''''''''''''''''''' The `base_urls` parameter specifies the domain and prefix url for the static tree, in which both the production and development resources must be available and accessible from the web. Only the resources that match this url, will be transformed. It is also possible to specify a list of urls for this, in which case any of the urls will be considered for matching:: base_urls = http://127.0.0.1:6543/ http://localhost:6543/ http://foo.bar/ One specific case when this is needed, if the local resources may have a different domain than the base url of the page in which they are referred from. In this case you simply list all the possible urls you may want to use during development. Specifying more does not do any harm, besides making the designated resources local. allow_request_url: disabling to consider the url host of the current request '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' `allow_request_url` is by default True, meaning that *both* the domains from `base_urls` *and* the domain of the current request url designate the locality of resources. It is possible to specify `allow_request_url` as False, in which case, the use of the request url is prohibited, and only the declaration `base_urls` will be considered. This would be useful, if your portal is not at the root of domain, for example, it is served from the following root url:: http://127.0.0.1:6543/myportal/... In this case you would use the following specification, which will cause `url_prefix` to be applied *after* the prefix you specify from `base_urls`:: allow_request_url = false base_urls = http://127.0.0.1:6543/myportal http://localhost:6543/myportal http://foo.bar/myportal Specifying `allow_request_url = false`, without also specifying `base_urls` to a sensible value, does not make sense, although it will not yield an error. The middleware will simply do nothing, not being able to identify any url as local. Specifying the static tree containing the resources """"""""""""""""""""""""""""""""""""""""""""""""""" Each resource that `gf.rejuice` handles is part of a static tree. Both the development, and the production resources are in some static tree. This tree resides somewhere on the filesystem, and the application also publishes this tree somewhere in its url space. The following parameters make it possible to define this tree. Both `url_prefix` and `filepath` need to be defined together. url_prefix: specifying the url prefix of the static tree root ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' The `url_prefix` parameter is an url path segment that defines where the tree root is located in the application's url space:: url_prefix = /static The above example defines that the static tree url is:: http://127.0.0.1:6543/static The `/` in the beginning (or end) of the prefix is optional, that is, the following parameters have an equivalent meaning:: url_prefix = /static/path :: url_prefix = static/path :: url_prefix = /static/path/ filepath: specifying the static tree root in the filesystem ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' The application uses the`filepath` parameter to locate the static tree on the filesystem. This must be the same static tree that is published under the `url_prefix` parameter described earlier:: filepath = views/static If the path is a relative path, (does not starts with a `/`), then the path will be interpreted relatively from the ini file that holds the declaration. It is also possible to use an absolute file path:: filepath = /Path/To/my/static/tree It is also possible to specify a file path relative from a python egg:: filepath = egg:my.package/views/static The above declaration will locate the import location of the `my.package` egg, and locate the tree from there, using the path section that follows the egg's name. The parameters `url_prefix` and `filepath` need to be defined together. Specifying and extending the composition of resources """"""""""""""""""""""""""""""""""""""""""""""""""""" `gf.rejuice` provides a simple and flexible way to define how the production (compressed) resources are produced from the development resources (the original sources). Specifying the composition of resources ''''''''''''''''''''''''''''''''''''''' A list of javascript or css resources can be specified from the configuration file. Anything ending with `.js` or `.css` will be considered to be a resource:: default.min.js = default.js min/default.min.css = custom/default.css Each line contains two values here. The value on the left of the equal sign specifies the path of the compressed, minified resource (the result of compression). The right value specifies the path of original, uncompressed source (the origin of compression). All file paths are applied relatively from the root of the static tree, specified by `url_prefix` and `filepath`. Derive a resource from a list of resources '''''''''''''''''''''''''''''''''''''''''' It is also possible to produce one resource from a list of resources, which, in that case will be compressed and concatenated to a single resource. The configuration allows a list of resources in multiple lines:: default.min.js = dependency.js dependency2.js default.js min/default.min.css = a/css1.css b/css2.css custom/default.css Juicer's scheme for specifying resource dependencies from inside the resources (by using `@import` from css files and `@depends` from commented parts of javascript files) is also taken into consideration. But if such dependencies are specified from the resources themselves, then they do not need to be (should not be) listed from the configuration section as well. extend_resources: adding more static trees to the configuration ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' For more flexibility, it is possible to define the resource composition from a separate configuration file section. In this case the filter section will refer to this section with the `extend_resources` parameter:: [filter:rejuice] use = egg:gf.rejuice#develjuice base_urls = http://127.0.0.1:6543/static extend_resources = juice_resources [juice_resources] url_prefix = /static filepath = views/static # Resource composition: default.min.js = default.js min/default.min.css = custom/default.css The `extend_resources` option selects a section in the ini file, that configures the actual resources to be considered for the transformation. Instead of a single section, a list of sections can also be specified:: extend_resources = juice_resources_1 juice_resources_2 Extending resources from another configuration file ''''''''''''''''''''''''''''''''''''''''''''''''''' Following the same syntax that `PasteDeploy` uses for extending its configuration, it is also possible to extend the resources from a section residing in an arbitrary .ini file:: extend_resources = config:another_config_file.ini#juice_resources This makes it possible to separate the resource configuration for `gf.rejuice` into a separate file. In the file `my-devel.ini` the following could be entered:: [filter:rejuice] use = egg:gf.rejuice#develjuice base_urls = http://127.0.0.1:6543/static extend_resources = config:another_config_file.ini#juice_resources While the referred file `another_config_file.ini` could look like this:: [juice_resources] url_prefix = /static filepath = egg:my.package/views/static # Resource composition: default.min.js = default.js min/default.min.css = custom/default.css The config: prefix makes it possible to refer a configuration file relatively from the location of the current configuration file:: extend_resources = config:another_config_file.ini#juice_resources Alternately, an absolute file path can be used as well:: extend_resources = config:/Path/To/my/place/another_config_file.ini#juice_resources Finally, the configuration file can be selected relative from a python egg import:: extend_resources = egg:my.package/path/to/another_config_file.ini#juice_resources Instead of a single extension section, a list of extension sections can also be specified:: extend_resources = base_resources more_resources egg:my.package/path/to/another_config_file.ini#juice_resources_1 egg:my.package/path/to/another_config_file.ini#juice_resources_2 Putting the resource declarations to a separate file makes it easier to manage them. The configuration file behaves just as expected from python configfiles or Paster configuration files. The `[DEFAULT]` section can specify a default value for the `url_prefix` and the `filepath` parameters, which then each section in the same file inherits, or can overwrite:: [DEFAULT] url_prefix = /static filepath = egg:my.package/views/static [juice_resources_1] default.min.js = default.js min/default.min.css = custom/default.css [juice_resources_2] some_other.min.js = some_other.js ... [juice_resources_3] url_prefix = /static2 filepath = egg:another.package/views/static different.min.js = different.js ... It is also possible, just as in every python config file, to use `%(varname)` style interpolation anywhere inside the declarations. `%(here)` containing the full path of the current ini file, as well as `%(__name__)` containing the name of the current section, are also available, just as usual in the Paster configuration files. Importing resources from another static tree '''''''''''''''''''''''''''''''''''''''''''' It is an important need to construct resources from another static tree than the one the final resource will be located in. Important use cases are: reusing the concatenation scheme of a resource defined elsewhere, or, making a site resource out of resources produced by separate javascript checkouts, or python eggs. A resource originating from another section can be specified by providing the section name, following the resource path by one or more spaces:: default.min.js = default.js othersection In a more realistic example, the resource would typically be produced from a list of resources from separate sections:: [main] url_prefix = /static filepath = egg:my.site/views/static default.min.js = jquery-1.4.4.min.js jquery_section jquery-ui-1.8.min.js jquery-ui_section default.js extend_resources = jquery_section jquery-ui_section [jquery-section] url_prefix = /static-jquery filepath = egg:my.jquery/static [jquery-ui_section] url_prefix = /static-jquery-ui filepath = egg:my.jqueryui/static Note that all the sections used as an import source, *must* also be listed in the `extend_resource` parameter. Failing to do so, will result in an error message. As seen from the above example, the resources imported from another section (`jquery-1.4.4.min.js`, and `jquery-ui-1.8.min.js` in this case) need not be specified as resources in that section. They just need to be present in the source tree. If however the imported resources are not themselves compressed, they will be. Also, if they contain further dependencies specified by `@import` or `@depends`, these dependencies will properly be included in the resulting resource. If the referenced sections contain further resource concatenation rules, they will be processed as well:: [main] url_prefix = /static filepath = egg:my.site/views/static default.min.js = jquery-1.4.4.min.js jquery_section jquery-ui-1.8.min.js jquery-ui_section default.js extend_resources = jquery_section jquery-ui_section [jquery-section] url_prefix = /static-jquery filepath = egg:my.jquery/static jquery-1.4.4.min.js = jquery.1.4.4.js [jquery-ui_section] url_prefix = /static-jquery-ui filepath = egg:my.jqueryui/static jquery-ui-1.8.min.js = ui/jquery.ui.core.js ui/jquery.ui.widget.js ui/jquery.ui.mouse.js ... ui/jquery.effects.transfer.js What will result from this configuration, that the middleware will explode every single resource to its development resources, recursively. Also, running `bin/rejuice myconfig.ini main` will produce all resources specified in all sections, providing a complete update based on the original sources. To make the example even more realistic, let us suppose that each static tree has a standalone configuration as well, which are referred from the configuration of the site. `site.ini`:: [main] url_prefix = /static filepath = views/static default.min.js = jquery-1.4.4.min.js egg:my.jquery/juicer.ini#jquery_section jquery-ui-1.8.min.js egg:my.jqueryui/juicer.ini#jquery-ui_section default.js extend_resources = egg:my.jquery/juicer.ini#jquery_section egg:my.jqueryui/juicer.ini#jquery-ui_section `juicer.ini` in `my.jquery` egg:: [jquery-section] url_prefix = /static-jquery filepath = views/static jquery-1.4.4.min.js = jquery.1.4.4.js `juicer.ini` in `my.jqueryui` egg:: [jquery-ui_section] url_prefix = /static-jquery-ui filepath = views/static jquery-ui-1.8.min.js = ui/jquery.ui.core.js ui/jquery.ui.widget.js ui/jquery.ui.mouse.js ... ui/jquery.effects.transfer.js Localizing images for imported css resources '''''''''''''''''''''''''''''''''''''''''''' A valuable feature of `gf.rejuice` is the support of concatenation a single css resource from foreign css resources, that is normally not supported by Juicer and hard to be achieved by Juicer alone. Css resources contain `url(...)` directives to refer to other resources, typically images. While Juicer takes care of rewriting the urls in these directives, so that the original resources can be located from the compressed css, the scheme will stop working as soon as a css file is composed from many different source trees or eggs. To support this use case, `gf.rejuice` copies all images referenced in the compressed css file, that reside in different source trees. This only happens if a css image compresses from any resource from a section which has a different `filepath` than the target section. For example, in case of a following configuration:: [main] url_prefix = /static filepath = egg:my.package/views/static default.min.css = other_package.css other_section default.css extend_resources = other_section [other_section] url_prefix = /other-static filepath = egg:other.package/static if we run `bin/rejuice myconfig.ini main`, the result will be a `default.min.css` file, and a `default.min.css.images` directory, created in the same directory next to each other. The `default.min.css.images` directory will contain a copy of all the resources that `default.min.css` references. As a consequence of this, any other sections than the main section, are only needed for development mode while the middleware resolves urls to the original sources, and for the time when the compression is done by running `bin/rejuice`. Following that, the other trees can be brought to offline as they are not needed to be present in the production website at all, that operates solely on the compressed resources. This makes it possible to establish a development scheme, when the production websites contains the compressed resources only, and the library dependencies, present in checked-out static trees or development eggs, are only present during the time of development. Example use cases ----------------- XXX
About
No description, website, or topics provided.
Resources
License
Unknown, GPL-2.0 licenses found
Licenses found
Unknown
LICENSE-BSD.txt
GPL-2.0
LICENSE-GPLv2.txt
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published