Relative include from a parent template breaks with module_directory

[sqlalchemy-bot]

Migrated issue, originally created by Anonymous

When the following two lines are used in concert, genshi exits with "OSError: [Errno 13] Permission denied: '/common'"

lookup    = TemplateLookup(directories=[fldr],
        default_filters=['unicode', 'h'],
        module_directory='/tmp/mako_modules')



<%include file="../../common/ficfan_common.css.incl" />

I've tracked the problem down to line 210 of Template.init() which naively joins module_directory and self.uri.

The solution I see being most feasible is to extend the policy embodied in lines 207-208 (if u[0] == '/', strip it) by treating self.uri as relative to the current template's path.

In my example, that would resolve to "/tmp/mako_modules/common/ficfan_common.css.incl" rather than "/common/ficfan_common.css.incl".

---

Attachments: testcase_boostrapper.py

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

sorry, I need a lot more detail here. I've created a test scenario exactly like yours which includes a similar subdirectory layout and all module files are written as they are supposed to be - when you say "extend the policy embodied", I've no idea where exactly this policy is to be extended ? There's no stack trace here so I actually am not sure what file cannot be written, I would assume the module file but that's not the case AFAICT.

A run of my test illustrates the following file layout generating the subsequent module layout:

classics-MacBook-Pro:mako_test classic$ find .
.
./bat
./bat/bar
./bat/bar/file.html
./foo
./foo/include.html
./test.py


classics-MacBook-Pro:mako_test classic$ find /tmp/mako_test/
/tmp/mako_test/
/tmp/mako_test//bat
/tmp/mako_test//bat/bar
/tmp/mako_test//bat/bar/file.html.py
/tmp/mako_test//bat/bar/file.html.pyc
/tmp/mako_test//foo
/tmp/mako_test//foo/include.html.py
/tmp/mako_test//foo/include.html.pyc

file.html:

`
test file.

<%include file="../../foo/include.html" />

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

Replying to [ticket:174 guest]:

I've tracked the problem down to line 210 of Template.init() which naively joins module_directory and self.uri.

also, are we talking about the same code here? the stripped "u" in question is exactly what's being joined:

            elif module_directory is not None:
                u = self.uri
                if u[0] == '/':
                    u = u[1:]
                path = os.path.abspath(
                        os.path.join(
                            os.path.normpath(module_directory), 
                            os.path.normpath(u) + ".py"
                            )
                        )

and that approach goes back at least as far as Mako 0.3.1. Is it possible your Mako has been locally modified ?

[sqlalchemy-bot]

Anonymous wrote:

Today was a very busy day and the next few days may be too, but I'll write myself a TODO note and get you a clear test case against a fresh pip install and a clearer explanation as soon as I can spare an hour.

[sqlalchemy-bot]

Anonymous wrote:

I forgot to say it in the description for that attachment, because I'm dead tired and not used to Trac, but all you do is run that testcase on a machine with pip installed.

It'll call tempfile.mkdtemp(dir=os.getcwd()) and then set up a reusable test environment inside that, complete with virtualenv and a copy of itself.

You can then re-run the test in that environment (complete with any modifications you made) using --testdir=PATH.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

Oh. Heh. The bug here is that Mako is letting you do something its not designed to do. You're setting your template root way into the directory tree, then trying to include a file that's outside of it using ".." to trick it. That wasn't made clear in your initial ticket where you just called it "fldr".

The "uri" of the Template shouldn't ever have any ".." in it - by the time the include tag builds up the Template the URL should be absolute. Otherwise if you have ten different templates that all include "common.incl" using a different relative scheme then you have "common.incl" in the template lookup ten times with different uris.

The correct way to load includes outside of the directory tree is to use a second path within the lookup, and include templates relative to that.

That it can very easily write the .py file all over the place here is such a big deal I may have to put a hard block on this and just release. It'll break some setups that are doing the same thing here but I almost feel like that's just how it has to be. Because it's writing files arbitrarily with this practice.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

5cd508f

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• set milestone to "0.5"
• changed status to closed

[sqlalchemy-bot]

Anonymous wrote:

Sounds good to me.

I sort of realized that was probably the case shortly after I posted the bug report (and solved it by using os.path.relpath to make os.walk(root) output relative to root so I could create one TemplateLookup instance for my entire static templating run).

My primary concern is ensuring that Mako can't unexpectedly put junk in unexpected places on my filesystem.