permissions of compiled templates

[sqlalchemy-bot]

Migrated issue, originally created by Anonymous

Mako creates the files for compiled templates using mkstemp, which restricts permissions to the current user. In our setup we need group permissions and now monkey patched mako.template._compile_module_file to run os.chmod(outputpath, 0664) after the call.

It could be nice to have a general fix of this problem similar to the solution for directories in Ticket #101.

Our use case: We have a setup where web tests are run under one user and apache runs under a different user. Both users are in a unix group, which is also the owner of the folder for compiled templates, which additionally has the sticky bit set (chmod g+s).

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

OK I frequently get confused on these issues, what I'm seeing is that the usage of mkstemp is only for writing out the file. The actual file that's used by the running of the template is the shutil.move() part, which moves the generated file to the ultimate destination (this is all of course for atomicity). The move isn't taking advantage of the sticky bit you have there ?

I can't exactly hardcode os.chmod in there since it isn't platform agnostic. I'd want to give you just a writer() hook that just goes into _compile_module_file and then you can do what you want. something like this:

diff -r 3266c0160d9103703400123b81a713db21ccd5be mako/lookup.py
--- a/mako/lookup.py	Wed Jan 11 19:25:46 2012 -0500
+++ b/mako/lookup.py	Sun Jan 15 19:44:45 2012 -0500
@@ -160,6 +160,7 @@
                         cache_url=None, 
 
                         modulename_callable=None, 
+                        module_writer=None,
                         default_filters=None, 
                         buffer_filters=(), 
                         strict_undefined=False,
@@ -195,6 +196,7 @@
             'encoding_errors':encoding_errors, 
             'input_encoding':input_encoding, 
             'module_directory':module_directory, 
+            'module_writer':module_writer,
             'cache_args':cache_args,
             'cache_enabled':cache_enabled, 
             'default_filters':default_filters, 
diff -r 3266c0160d9103703400123b81a713db21ccd5be mako/template.py
--- a/mako/template.py	Wed Jan 11 19:25:46 2012 -0500
+++ b/mako/template.py	Sun Jan 15 19:44:45 2012 -0500
@@ -154,6 +154,7 @@
                     module_filename=None, 
                     input_encoding=None, 
                     disable_unicode=False,
+                    module_writer=None,
                     bytestring_passthrough=False, 
                     default_filters=None, 
                     buffer_filters=(), 
@@ -188,6 +189,7 @@
         self.disable_unicode = disable_unicode
         self.bytestring_passthrough = bytestring_passthrough or disable_unicode
         self.strict_undefined = strict_undefined
+        self.module_writer = module_writer
 
         if util.py3k and disable_unicode:
             raise exceptions.UnsupportedError(
@@ -276,7 +278,8 @@
                             self, 
                             open(filename, 'rb').read(), 
                             filename, 
-                            path)
+                            path,
+                            self.module_writer)
             module = imp.load_source(self.module_id, path, open(path, 'rb'))
             del sys.modules[self.module_id]
             if module._magic_number != codegen.MAGIC_NUMBER:
@@ -284,7 +287,8 @@
                             self, 
                             open(filename, 'rb').read(), 
                             filename, 
-                            path)
+                            path,
+                            self.module_writer)
                 module = imp.load_source(self.module_id, path, open(path, 'rb'))
                 del sys.modules[self.module_id]
             ModuleInfo(module, path, self, filename, None, None)
@@ -543,7 +547,7 @@
     exec code in module.__dict__, module.__dict__
     return (source, module)
 
-def _compile_module_file(template, text, filename, outputpath):
+def _compile_module_file(template, text, filename, outputpath, writer=None):
     identifier = template.module_id
     lexer = Lexer(text, 
                     filename, 
@@ -563,17 +567,20 @@
                                 disable_unicode=template.disable_unicode,
                                 strict_undefined=template.strict_undefined)
  
-    # make tempfiles in the same location as the ultimate 
-    # location.   this ensures they're on the same filesystem,
-    # avoiding synchronization issues.
-    (dest, name) = tempfile.mkstemp(dir=os.path.dirname(outputpath))
- 
     if isinstance(source, unicode):
         source = source.encode(lexer.encoding or 'ascii')
+
+    if writer:
+        writer(source, outputpath)
+    else:
+        # make tempfiles in the same location as the ultimate 
+        # location.   this ensures they're on the same filesystem,
+        # avoiding synchronization issues.
+        (dest, name) = tempfile.mkstemp(dir=os.path.dirname(outputpath))
  
-    os.write(dest, source)
-    os.close(dest)
-    shutil.move(name, outputpath)
+        os.write(dest, source)
+        os.close(dest)
+        shutil.move(name, outputpath)
 
 def _get_module_info_from_callable(callable_):
     return _get_module_info(callable_.func_globals['__name__'])

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• set milestone to "0.5"

[sqlalchemy-bot]

Anonymous wrote:

Since the temporary file is already created in the outputpath, the group is set correctly at creation time due to the sticky bit of the outputpath on our server.

A (non portable) configuration option to optionally set the permissions after creation to a given value would be sufficient in our case. A module_writer as presented would allow us to fix it ourselves. Whatever works for you :). I would opt for giving the writer variable the same name everywhere, e.g. also call the last arg to _compile_module_file module_writer instead of writer.

Thx!

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

I like that, you care about the names within the internals :). The =None isn't needed either. My giving you a callable here is my way of hedging on, is allowing "os.chmod" too specific ? Am I just going to be adding four more flags for other people's issues here ? If I just do a callable then I don't need to be concerned about it. I really don't like adding new flags that just fix one specific problem for only one reporter (and we're six years into it...), though I suppose others might have hit this and then just opened up the permissions on their directory. The change is in 9e134cb I hope it works for you !

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

Milestone 0.6 deleted

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• removed milestone (was: "0.6")