Skip to content
This repository

problem with date format #17

Closed
wants to merge 4 commits into from

2 participants

jerzyk William
jerzyk

when you will set locale that is different than en_US or C, there may be an issues related to date formatting.

added some commits September 08, 2010
William

I'm agreed that current date format is not http compliant because it's the unix time format. But could you explain why type of issues did you have ?
Anyhow to be consistent your proposed change requires changes in the Staticfile class too (maybe in other places too).

jerzyk

Basically, actual method is strictly locale dependent, so if your system has different locale than C or en_US all Date and Last-Modified are messed up, in effect no cache or anything that is using headers will not work.

E.g.

  • static files will always be served fully instead utilising browser cache
  • varnish will report strange errors

I've found date only in those two files:

  • base.py - main Date header and cookies
  • views.py - last-modified header
William

Currently Fapws is using the unix time format: nbr of seconds since /01/01/1097. I don't understand how "local" will have an impact on that.

Regarding staticfiles ... If you use the Staticfile object, you will see that it return lot of 304 http messages. I've tested it with lot of different browser and they took the files from their cache really fine.

Concerning "varnish" ... Their site sounds down for the moment, I cannot thus check what it is. But if such tool cache files and required a valid date format, you will have a problem with current fapws.
Do not forget that, in such a case, you could easily solve this by overwriting the http header before sending it.

jerzyk

sorry, maybe I was not clear:

In [1]: import datetime
In [2]: datetime.datetime.now().strftime('%a, %d %b %Y %H:%M:%S GMT')
Out[2]: 'Fri, 10 Sep 2010 13:39:40 GMT'
In [3]: import locale
In [4]: locale.setlocale(locale.LC_ALL, 'pl_PL.UTF-8')
Out[4]: 'pl_PL.UTF-8'
In [5]: datetime.datetime.now().strftime('%a, %d %b %Y %H:%M:%S GMT')
Out[5]: 'ptk, 10 wrz 2010 13:39:50 GMT'

If you have in the code (e.g. django app) locale change, it will mess-up your headers as strftime
is locale dependent.

This applies in 3 places (where RFC 1123 formatted date is required).

I've given a varnish as an example, but this is same for other caches, microsoft's or w3cache.

When header is incorrect - is is being ignored.

William

Fapws is using the standard date format recognise by HTTP RFC.

Concerning Staticfile, it's better to use Etag than date. This is what I've just changed.
Thanks to check the change of today.

W.

ps:
thanks to confirm if I can close the issue

jerzyk

Date is still used for cookie..

William

Please can you be much more precise ?

If I read line 136 of base.py, I'm using the standard date format too.

jerzyk

so, again - no, format is not standard, at least not all the time. This code will work, but with the assumption that system locale is C or en_US/UK and nobody changes locale during web application execution.

If you are using multiple languages (thus multiple locales) using strftime will lead to non-RFC date format.

This is happens because strftime formatting function are locale bound, so to display date&time they are using values defined by the locale.

E.g. "wed" will become "śro" in pl_PL locale etc. - so date will not be "standard date format".

So this code works with the assumption, that C locale is a default all the time, which it shoudn't.

In the proposed code, I'm using function that is independent of actual locale settings, so it will work despite locale change.

This is common discussion of you will search google.

One of the examples is gunicorn, their approach was to define own function to format date. Look at http://github.com/benoitc/gunicorn/blob/master/gunicorn/util.py

My suggestion was to use standard library, but choice is yours.

Regarding other issue - using ETags, yes it is fine, but if you will put long time expire headers for the static files, browser will not even try to load a file (to check if it is changed) so yes, it will work, but is not optimal - browser will not use cache in 100% - it will try to get a resource and use cache if response was "not modified". But with additional expire header - connection will not be established. For the high-traffic sites, this will reduce significantly, total connections made to the site.

best regards,
Janusz

William

Since Lighttpd is also using strftime, does this means that lighhtpd has also a date problem ?
Concerning your proposed solution ... it does not include a solution for the C part of the application.

jerzyk

yes and no :)
yes, because if there will be call to the locale() in the lighttpd process, then date will be affected

no, because there are no changes to locale in main code or modules (at least in built-in modules) and additionally, there is a setting to force the locale to "C" at the beginning of the code (even with nice comment ;))

"src/server.c" line 546:
/* for nice %b handling in strfime() */
setlocale(LC_TIME, "C");

With fapws3, there are no guarantee that user code (which is executed as part of the main process) will not change a locale.

For the C part of the application, a solution can be to store current locale setting, set it to "C", then prepare all date headers, then set it back.

Not sure if storing and setting it back will be needed (didn't track an execution flow), but it better to set it back, as some of the application will change locale only on the code load, then assuming it is being set.

regards,
Janusz

William

Thus best option would be to create a generic variable (in config.py) where the users set the locale.
In python, we can set it by using locale.setlocale() in python; and in C, if not heritate from python, by using setlocale().
A comment will explain that changing it will break the RFC recommendations.
But if someone want absolutely change it, then why not.

jerzyk

In my opinion, this is not the best option, simply because it will not work out-of-the-box as a replacement of any other comparable server.

Plus, this is not solving an issue, but trying to force users to do something in a very specific way.

If you really, really, really do not want to put small extra code in python (like e.g. gunicorn), so add this:
tmp_locale = locale.get_locale()
locale.set_locale('C')
date_str = datetime.datetime.now().strftime(....)
locale.set_locale(tmp_locale)

pretty simple, no additional configuration, no additional lines of manual to read (and you know that nobody reads manuals ;))

I still think, that adding this simple function to format date, and then set headers in python (as it was) will be simplest and most portable. This function can be used to format date in "Date", "Last-modified" and "Expire" headers and cookies. One solution for all of them.

J.

William

OK. In C, we could do something like this.

char *days_names[] ={ "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
char *month_names[] ={ "Jan", "Feb", "Mar", "Apr", "May", "Jun","Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };
strftime(outstr, date_len+1, "---, %d --- %Y %H:%M:%S GMT", tmp);
memcpy(outstr, days_names[tmp->tm_wday], 3);
memcpy(outstr+8, month_names[tmp->tm_mon], 3);

But, now I'm looking a way to send a python datetime object to such C function and convert it to time_t. Should not be that difficult.

jerzyk

I'm confused ;)
maybe I'm missing something, but in the C code, there is only one place where you are using date formatting - response header, but.. this was few changesets before in the python code - that means there is no need to process date in the C code at all

William

the last commit should solve the date problem.
I prefer to have it in C for performance reasons.

You are right ... you can overwrite what fapws is doing within your python code and before sending the response to the browser. Thus indeed, you could overwrite the date in an another format; depends what you want ;-).

Could you test and confirm I can close this issue ?

W.

William
Owner

Could you confirm that I can close the issue ?

Thanks

William
Owner

issue not reproducable

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 4 unique commits by 1 author.

Sep 08, 2010
fix date formatting adbf160
add missing import 20a12a9
fix datetime import 2a5284b
Sep 15, 2010
Merge branch 'master' of git://github.com/william-os4y/fapws3
Conflicts:
	fapws/base.py
6c31fc8
This page is out of date. Refresh to see the latest.
6  fapws/base.py
@@ -21,8 +21,8 @@
21 21
 except ImportError:
22 22
     import StringIO
23 23
 import traceback, sys, string
24  
-
25  
-
  24
+from wsgiref.handlers import format_date_time
  25
+from time import mktime
26 26
 
27 27
 import config
28 28
 
@@ -133,7 +133,7 @@ def set_cookie(self, key, value='', max_age=None, expires=None, path='/', domain
133 133
             if isinstance(expires, str):
134 134
                 self.cookies[key]['expires'] = expires
135 135
             elif isinstance(expires, datetime.datetime):
136  
-                expires = expires.strftime(config.date_format)
  136
+                expires = format_date_time(mktime(expires.timetuple()))
137 137
             else:
138 138
                 raise CookieError, 'expires must be a datetime object or a string'
139 139
             self.cookies[key]['expires'] = expires
4  fapws/contrib/views.py
@@ -15,6 +15,8 @@
15 15
 import mimetypes
16 16
 import os.path
17 17
 import time
  18
+from wsgiref.handlers import format_date_time
  19
+
18 20
 
19 21
 class Staticfile:
20 22
     """ Generic class that you can use to dispatch static files
@@ -42,7 +44,7 @@ def __call__(self, environ, start_response):
42 44
             #print "NEW", environ['fapws.uri']
43 45
             ftype=mimetypes.guess_type(fpath)[0]
44 46
             headers.append(('Content-Type',ftype))
45  
-            headers.append(('Last-Modified',fmtime))
  47
+            headers.append(('Last-Modified',format_date_time(fmtime)))
46 48
             headers.append(('Content-Length',os.path.getsize(fpath)))
47 49
             start_response('200 OK', headers)
48 50
             return f
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.