Moving assets out of the /static directory, possible? :) #35

Closed
fgimian opened this Issue Jan 3, 2013 · 15 comments

Projects

None yet

3 participants

@fgimian
fgimian commented Jan 3, 2013

Hello there, I think that webassets and the related Flask module are awesome and just had a few questions about the way files are read and structured within a Flask project.

Suppose we wish to keep track of 3 item types:

  1. Non-compiled assets (e.g. LESS CSS, Stylus, Coffeescript .etc)
  2. Compiled assets (e.g. CSS, JS)
  3. Third party libraries used in a project (e.g. jQuery, Twitter Bootstrap)

I think a suitable directory structure would probably look something like this:

flask_project/assets/less/...   <- (1) Non-compiled assets
flask_project/assets/coffee/... <- (1) Non-compiled assets
flask_project/static/js/...     <- (2) Compiled assets
flask_project/static/js/...     <- (3) 3rd Party Libraries
flask_project/static/css/...    <- (2) Compiled assets
flask_project/static/css/...    <- (3) 3rd Party Libraries

I have been experimenting with Flask-Assets and can't seem to find a way to achieve this without doing something like this in the template:

{% assets filters="less", output="css/style.css",
          "../assets/less/style.less", "../assets/less/import.less" %}
    <link rel="stylesheet" type="text/css" media="screen" href="{{ ASSET_URL }}" />
{% endassets %}

I also attempted to use the ASSETS_LOAD_PATH config item but didn't have any luck.

Is there any way to specify the source directory independently from the destination directory?

Also another little question, is the cache really needed? For example, I pictured that it would work as follows.

  1. I update my non-compiled source
  2. The hash is saved in a dict (or similar)
  3. An output file is generated
  4. Next time I refresh the page, the hash is calculated against each file and compared to the value in the dict
  5. If it has changed, repeat steps above

Please let me know your thoughts and if I'm missing something :)

Thank you so much and keep up the great work!
Fotis

@fgimian
fgimian commented Jan 3, 2013

Just thought I'd provide further info about my troubles with using load_path:

app = Flask(__name__)
assets = Environment(app)

# Each of these load_path settings were tested
assets.append_path('/home/fots/flask-less-assets/assets/')
assets.append_path('/home/fots/flask-less-assets/assets')
assets.append_path('assets')

Each of these 3 cases resulted in the following exception:

Traceback (most recent call last):
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/fots/flask-less-assets/app.py", line 13, in hello
    return render_template('index.html')
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/templating.py", line 125, in render_template
    context, ctx.app)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/templating.py", line 107, in _render
    rv = template.render(context)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/fots/flask-less-assets/templates/index.html", line 4, in top-level template code
    {% assets filters="less", output="css/style.css",
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/ext/jinja2.py", line 181, in _render_assets
    urls = bundle.urls(env=env)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 681, in urls
    urls.extend(bundle._urls(env, extra_filters, *args, **kwargs))
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 644, in _urls
    return [self._make_output_url(env)]
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 606, in _make_output_url
    url = env.resolver.resolve_output_to_url(url)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask_assets.py", line 234, in resolve_output_to_url
    return self.resolve_source_to_url(None, target)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask_assets.py", line 197, in resolve_source_to_url
    return super(FlaskResolver, self).resolve_source_to_url(filepath, item)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/env.py", line 300, in resolve_source_to_url
    return self.query_url_mapping(filepath)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/env.py", line 232, in query_url_mapping
    needle = path.normpath(filepath)
  File "/usr/lib/python2.7/posixpath.py", line 318, in normpath
    initial_slashes = path.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

And this case caused a different exception to occur:

app = Flask(__name__)
assets = Environment(app)

assets.append_path('/assets')

Which produced this exception:

Traceback (most recent call last):
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/fots/flask-less-assets/app.py", line 13, in hello
    return render_template('index.html')
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/templating.py", line 125, in render_template
    context, ctx.app)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/flask/templating.py", line 107, in _render
    rv = template.render(context)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/fots/flask-less-assets/templates/index.html", line 4, in top-level template code
    {% assets filters="less", output="css/style.css",
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/ext/jinja2.py", line 181, in _render_assets
    urls = bundle.urls(env=env)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 681, in urls
    urls.extend(bundle._urls(env, extra_filters, *args, **kwargs))
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 643, in _urls
    *args, **kwargs)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 488, in _build
    if env.updater else True
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/updater.py", line 172, in needs_rebuild
    self.check_timestamps(bundle, env)
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/updater.py", line 152, in check_timestamps
    for item in iterator(env):
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/updater.py", line 149, in <lambda>
    (lambda e: map(lambda s: s[1], bundle.resolve_contents(e)), True),
  File "/home/fots/.virtualenv/webdev/lib/python2.7/site-packages/webassets/bundle.py", line 148, in resolve_contents
    raise BundleError(e)
BundleError: 'less/style.less' not found in load path: ['/assets']

I have dug through the source code but didn't have any luck determining the root cause, mostly because I don't believe I've understood the load_path configuration item correctly.

@miracle2k
Owner

The bug is here: https://github.com/miracle2k/flask-assets/blob/master/src/flask_assets.py#L234 - the method doesn't work if None is passed.

As a temporary workaround, you can set an explicit output directory, for example assets_env.directory = app.static_folder

@miracle2k
Owner

With regards to the cache: webassets has the env.auto_build option, which will look at the timestamps of files to determine what has changed and needs to be recompiled. The cache is an addition to that to make the compilation faster. Imagine you have a bundle with 10 files A to J, and each of them should be passed through a Jinja 2 template pre-processor.

Without the cache, if you change file A, in other to produce a new output file, all 10 need to be reprocessed by the Jinja filter. With the cache, only A needs to go through Jinja, and the rest can be read from the cache.

@fgimian
fgimian commented Jan 3, 2013

Thank you so much for the quick reply and really appreciate the explanation of caching and its requirement, that makes a lot of sense.

I am happy to confirm that the following code works as you suggested, although I also had to set the url property or I would get an Environment exception:

app = Flask(__name__)
assets = Environment(app)

assets.url = app.static_url_path
assets.directory = app.static_folder
assets.append_path('assets')

I'll be working with Flask-Assets further in the next few days. I think it's a really important extension to Flask for modern web development. Frameworks like Rails, Django (via compressor) and Play provide a way to use languages like LESS and Coffeescript and I really love the fact that your extension provides a similar, well-implemented facility for Flask.

Thanks a lot for your help! I'll happily test your bug fix implementation when ready :)

@miracle2k miracle2k closed this in f86f45b Jan 3, 2013
@cburmeister

is this not possible with init_app ? I cannot get around the assets instance not bound to an application, and no application in current context.

  1. app/assets.py
from flask.ext.assets import Bundle, Environment
assets = Environment()
  1. app/__init__.py
from app.assets import assets
app = Flask(__name__)
assets.init_app(app)

Where should I be configuring assets with this pattern?

All I want is to move some javascripts/stylesheets out of the app/static folder and into assets/. Any advice is appreciated.

@miracle2k
Owner

No, init_app is a different thing entirely.

You either need to set a value for assets_env.directory (if source and output folder are supposed to be the same), or use assets_env.append_path (if you want a different source folder than the assets_env.directory output folder.

@miracle2k
Owner

Just to be clear, there should be no reason this does not work with init__app. The error message you are seeing is because in that mode, you need to have an application on the stack. I.e. something like:

with app.test_request_context():
     assets_env.do_something()
@cburmeister

Thanks for the reply, but i'm still unclear. In my pattern above, where should I be configuring assets_env?

@miracle2k
Owner

Like this:

from flask.ext.assets import Bundle, Environment
assets = Environment()
assets.directory = './assets'
@cburmeister
  1. app/assets.py
from flask.ext.assets import Bundle, Environment
assets = Environment()
assets.directory = './assets'
  1. app/__init__.py
from app.assets import assets
app = Flask(__name__)
assets.init_app(app)

results in:
RuntimeError: assets instance not bound to an application, and no application in current context

@miracle2k
Owner

When you do what? A full strack trace would be easiest.

@cburmeister

Here is the full stack trace:

(flask-bones)➜  flask-bones git:(master) ✗ python manage.py runserver
Traceback (most recent call last):
  File "manage.py", line 2, in <module>
    from app import create_app
  File "/Users/cburmeister/src/flask-bones/app/__init__.py", line 4, in <module>
    from app.assets import assets
  File "/Users/cburmeister/src/flask-bones/app/assets.py", line 21, in <module>
    assets.directory = './assets'
  File "/Users/cburmeister/.pythonbrew/venvs/Python-2.7.3/flask-bones/lib/python2.7/site-packages/flask_assets.py", line 270, in set_directory
    self.config['directory'] = directory
  File "/Users/cburmeister/.pythonbrew/venvs/Python-2.7.3/flask-bones/lib/python2.7/site-packages/flask_assets.py", line 103, in __setitem__
    self.env._app.config[self._transform_key(key)] = value
  File "/Users/cburmeister/.pythonbrew/venvs/Python-2.7.3/flask-bones/lib/python2.7/site-packages/flask_assets.py", line 260, in _app
    'and no application in current context')
RuntimeError: assets instance not bound to an application, and no application in current context
@miracle2k
Owner

The code that I gave you was indeed wrong, but it speaks to the problem you are having in general. This should really be the same as any other Flask extension in init_app mode:

app = Flask(__name__)
with app.app_context():
    # within this block, current_app points to app.
    assets.directory = 'foo'

With init_app(), you may use the same assets object with different applications, and it needs to know the app you are configuring, since it stores the configuration in the Flask app settings object.

See here:
http://flask.pocoo.org/docs/appcontext/#app-context

@fgimian
fgimian commented Feb 10, 2014

Hey mate, I apologise to bring this topic up again, but the url property is still causing dramas in the current version on Github. I still must add the following line of code to ensure that a custom asset load path works...

assets.url = app.static_url_path

Can this be repaired so that the workaround is not necessary (similar to how you fixed the directory property problem above)?

Thanks heaps!
Fotis

@miracle2k
Owner

I'm not sure if what you are talking about is the same problem, but then again, the above is a pretty long thread and I don't really remember what it is about.

Please open a new ticket with a summary of what you want to accomplish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment