Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Feature request: Show exact path where error occurs #12

Open
ckreutz opened this Issue Apr 12, 2012 · 7 comments

Comments

Projects
None yet
4 participants

ckreutz commented Apr 12, 2012

First of all thanks a lot for this brilliant module. It has helped us a lot with transparency projects such as this one: http://haushalt.frankfurt-gestalten.de/. Freeze is for us also a great test module to see where a page is broken. But the challenge is that often the error does not tell us where the problem occurs. For example in the above budget transparency page we have a template, which builds a thousand pages.

Therefore is it possible to extend the module, so it gives you a path, where the problem occurred? Thanks in advance for any hint or advise.

Collaborator

SimonSapin commented Apr 12, 2012

Hi. I’m glad to hear that this project is useful to you. The linked URL does not work for me. (The DNS name does not resolve.) Is it supposed to be already up?

As to your request: Frozen-Flask currently aborts completely if some response has a status code other than 200. Is that the kind of error where you would like more reporting? The exception message contains the status code and the URL. What information would you like to see added there? Maybe the endpoint + dict that generated the URL?

ckreutz commented Apr 12, 2012

Thanks for your quick reply!
Please try this url: http://frankfurt.offenerhaushalt.de instead.

Yes I was talking about when Frozen-Flask aborts completely.
For example I get an error such as this one:
ValueError: Unexpected status '404 NOT FOUND' on URL /example//foo/

Now it is obvious the url was build wrong in the first place, but I do not get information, which was the previous page that build the incorrect url.

Or this error:
werkzeug.routing.BuildError: ('show_page', {'page': 140, 'foo': 71}, None)

Here it seems some parameters are missing to build the correct URL, but again it is difficult to identify on which page that error has happened. In my case I have hundreds of pages from the same template using show_page(page,foo,bar), but the message does not tell me where it happens. Sometimes I can find the error when looking at the folder structure build so far, but in many cases that does not help either.

Collaborator

SimonSapin commented Apr 12, 2012

Ok, I understand the situation but what change exactly would you like to see? (Or make ;))

There are three situations where Frozen-Flask aborts:

  • One of your URL generator yields a malformed value
  • One of your URL generator yields an endpoint + a dict of values. Frozen-Flask calls url_for() on that, which fails because the endpoint or the set of keys is wrong.
  • Frozen-Flask makes a request to your application. The response has a 404, 500 or other status code that is not 200.

Now that MIME type mismatches are not fatal anymore (they’re just warnings), I think that these three are the only cases that can cause errors. Are there more?

As to what to do: there could be better error reporting for each of these case. What information do you think would be useful? I can think of: the URL generator function, the module it is from, and the exact value it yielded.

ckreutz commented Apr 13, 2012

"What information do you think would be useful? I can think of: the URL generator function, the module it is from, and the exact value it yielded."

For me the path to the page, where the error happened would be most valuable such as /foo/bar/1 At the moment I often go through the folder/file structure to see which page was latest build to identify the error. I am not sure this info helps, but I am afraid I do not know if that can be part of the url generator function.

The information of the path would be very valuable and would make Frozen-Flask also a perfect testing module for local development. If your app passes Frozen-Flask it does not have broken pages.

Chiming in here -- I'm getting 404s that are causing the app to abort as well. In my case, the URL it's picking up isn't a URL that should be live in the site. I"m not sure how this is getting picked up. If the message gave me more context, I might be able to locate it. Quite literally, that URL is not in my code when I look using Find/Grep However, of course it is coming from somewhere. Where?

After spending a few mins looking, I decided it wasn't worth my time, but then I can't tell Frozen to ignore that URL (not abort for this) either. So now I'm stuck. This is frustrating because now I must figure out how Frozen found this URL so that I can fix it or remove it. This could take hours. Can 404s, 403s, 500s, etc, be warnings? Maybe a config var like "WARN_ON_404=True" and WARN_ON_500=True", etc.

Like the first poster said -- where is this URL coming from? That's what we need to know to fix this.

Referring view function, URL / route
Erroring view function, URL / route

When freeze() crawls the site, it must have some kind of context -- like a page or view or URL that it's searching URLs within. Referring page maybe? If not, then maybe keep a stack of the previous two view functions or URLs called. That might give a hint where to look. A config var can turn this on/off perhaps.

I'd prefer the Referring URL / view though.

This would help immensely, as it isn't always as simple as just running frozen-flask on an existing flask site.
At the moment I'm getting random 404 errors during freezing for files that do exist, the solution is to re-run frozen-flask until it works. Like I said, random =P.
I haven't opened a bug because I can't track the logic down.
But a warning, rather than full exception, on 404 would be helpful.
It would also help to understand more about why the url failed, and where it was called from.
I could wrap my url generators with exception handlers, but its often on supporting assets, favicons, jpgs etc.

Here's an example exception:

2014-03-25T03:02:23.963722+00:00 app[worker.1]: Traceback (most recent call last):
2014-03-25T03:02:23.963722+00:00 app[worker.1]:   File "worker.py", line 38, in update_website
2014-03-25T03:02:23.963722+00:00 app[worker.1]:     flask.freeze()
2014-03-25T03:02:23.963722+00:00 app[worker.1]:   File "/app/sitebuilder/flask.py", line 110, in freeze
2014-03-25T03:02:23.963722+00:00 app[worker.1]:     freezer().freeze()
2014-03-25T03:02:23.963722+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask_frozen/__init__.py", line 149, in freeze
2014-03-25T03:02:23.963722+00:00 app[worker.1]:     new_filename = self._build_one(url)
2014-03-25T03:02:23.963722+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask_frozen/__init__.py", line 263, in _build_one
2014-03-25T03:02:23.963722+00:00 app[worker.1]:     % (response.status, url))
2014-03-25T03:02:23.963985+00:00 app[worker.1]: ValueError: Unexpected status '404 NOT FOUND' on URL /static/site//static/assets/BHUXYzq4C3H.jpg

The stack trace is useless, not to mention that asset shouldn't be causing a problem.
The double slash is also a pita for freezing to s3.

Collaborator

SimonSapin commented Mar 25, 2014

Hi everyone. I agree that printing more information when aborting would be very useful, but I’m not actively working on Frozen-Flask right now. I would however review a pull request for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment