Skip to content

Loading…

[HttpKernel] removed absolute paths from the generated container #10999

Closed
wants to merge 5 commits into from
@fabpot
Symfony member
Q A
Bug fix? yes
New feature? no
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets first step to resolve #6484, #3079, and #9238
License MIT
Doc PR n/a

This code was merged to 2.3 in (#10894, #10937, and #10979) but it caused too many problems and so was reverted. This PR targets master (aka 2.6) to get plenty of time to figure out the best way to implement this feature without too many BC breaks.

Some issues that should be taken care of before re-merging this into master: #10976, #10943, symfony/symfony-standard#659.

The biggest problem is the usage of dirname which is naive per the PHP documentation: "dirname() operates naively on the input string, and is not aware of the actual filesystem, or path components such as "..".")". So, basically, as we have now .. in the root and cache dirs, using dirname on them is not possible anymore. One solution would be to use dirname calls instead of adding .. specifically for the root and cache dir, but other paths could be used a problem as well. A better fix would be to check usage of all dirname calls and be sure that we operate on a safe path or call realpath when needed.

From the original PR:

This PR converts absolute paths to relative ones in the dumped container. The code is a bit "ugly", but it gets the job done and I'm not sure that there is a more elegant way without breaking everything.

fabpot and others added some commits
@fabpot fabpot [HttpKernel] removed absolute paths from the generated container 9477e96
@BenoitLeveque BenoitLeveque Fix "absolute path" when we look to the cache directory
We should get __DIR__ instead of __DIR__ . '/.'
47634cc
@fabpot fabpot fixed CS fc208ac
@artursvonda artursvonda Make rootPath part of regex greedy
- Fixes #10977
9da641d
@fabpot fabpot [HttpKernel] simplified some tests
d8cc22f
@mvrhov

@fabpot realpath won't work when the kernel will be inside phar archive.

A long time ago I solved the problem of absolute paths by defining a root dir of the application by a define inside of front controller. In the symfony's case the root dir would be the dir where composer.json resides.

@jameshalsall

@mvrhov That won't work so well when you're running console commands in the standard dist

@mvrhov

For the console command it could be setup inside command file.

@stof
Symfony member

The logic available in Composer might help to solve the issue of ..: https://github.com/composer/composer/blob/master/src/Composer/Util/Filesystem.php#L207-358

@jakzal
Symfony member

Practice of putting the cache folder outside of the project root directory is not uncommon. I think we should consider such scenarios (see #11011 for example).

@Axel29

Hello,

The bug seems to be corrected with Windows but not with Mac. I have this message with the config.php file:
"Your parameters.yml file has been overwritten with these parameters (in /path/to/symfony/app/cache/dev/../../config/parameters.yml):"

And when I try to create a bundle with console:
"Target directory [/path/to/symfony/app/cache/dev/../src]:"

Does anyone have the solution?

@dzuelke

Any chance to roll a release soon with the revert, @fabpot? We're seeing a lot of users hit it with 2.4.5.

@dmaicher

We ran into the same problem on 2.4.5. Now 2.4.6 has been released and the revert is included ;)

@fabpot fabpot added the HttpKernel label
@toretto460

Also the Heroku infrastructure seems to be affected by this issue, heroku/heroku-buildpack-php#64

@ricardclau

+1 on removing absolute paths from the files in the cache folder and make everything relative using __DIR__ where possible

If you want to build deb / rpm packages or similar, the absolute paths force you to run cache:warm --env=prod in every server you deploy to. And even if you use rsync or any similar method, if you are deploying to 150 servers this is a lot of useless work.

Ok, you can have a build server matching the same folders structure but this is a hacky solution

Having said that, agreed that there are many players and infrastructures to take into account so 2.6 or even 2.7 should be the target for that

@fabpot
Symfony member

Looking at this issue again, I think there is no easy fix. Using dirname() will indeed work but as noted by @jakzal, the cache dir can be outside of the root dir. In this case, using dirname() won't help.

I cannot see any way to make it reliable in all cases, so I propose to make this an opt-in feature. You would opt-in by defining a constant (something like SYMFONY_ROOT_DIR). When defined, Symfony would make paths relative to the constant.

We can add this easily in Symfony Standard Edition (in autoload.php to make it available for the Web and in the CLI):

// using realpath here is required
define('SYMFONY_ROOT_DIR', realpath(__DIR__.'/...'));
@fabpot
Symfony member

A refinement of the above could be to not require the constant when the cache dir is indeed under the root_dir.

@stof
Symfony member

@fabpot why do we need the constant in most cases ? We have $kernel->getRootDir() to give us the path, and it is available before working with the cache as it comes from the kernel. Then, we can make build a relative path. The logic has been implemented under MIT license in Composer already: https://github.com/composer/composer/blob/31eadc6920cd1866bc061fb0087798c37e2b7d14/src/Composer/Util/Filesystem.php#L300-382 We can borrow it.
The only case where we would face issues is when the path between your root folder and your cache folder changes between your different environments (locally or in prod) and you want to be able to copy the cache).

The real pain point is actually being able to identify all paths in the container needing to be rewritten, because not all the code is inside the kernel root dir. Or were you suggesting to have SYMFONY_ROOT_DIR being the root of the project itself, not the app/ folder ? In this case, it could indeed work easier to identify the paths, but it should have a different name to avoid confusion.

And rather than a constant, couldn't we use a method in the Kernel class ? This way, it would not require any change for existing projects (considering that the root of the project is %kernel.root_dir%/..

@jakzal jakzal referenced this pull request
Closed

Opcache + Twig Templates #12172

@dzuelke

The logic is here as well, right? Symfony\Component\Filesystem\Filesystem::makePathRelative()?

@eddiejaoude

Has this been resolved, as still as issue on Heroku deployment?

@dzuelke

Two steps that help mitigate this:

  1. make sure you heroku config:set SYMFONY_ENV=prod before pushing
  2. in composer.json's post-install-cmd list, move the clearCache command to the very end
@eddiejaoude

Thanks. I have step 1 already. Step 2 removed that error but now I get no routes are found, always 404 on Heroku, locally & amazon work fine.

@desyncr desyncr referenced this pull request in heroku/heroku-buildpack-php
Closed

Unable to deploy Symfony application #64

@nicolas-grekas

@eddiejaoude can you please test #12784 and see if it helps?

@fabpot
Symfony member

closing in favor of #12784

@fabpot fabpot closed this
@fabpot fabpot added a commit that referenced this pull request
@fabpot fabpot bug #12784 [DependencyInjection] make paths relative to __DIR__ in th…
…e generated container (nicolas-grekas)

This PR was merged into the 2.3 branch.

Discussion
----------

[DependencyInjection] make paths relative to __DIR__ in the generated container

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #6484, #3079, partially #9238, #10894, #10999
| License       | MIT
| Doc PR        | n/a

This is an alternative approach to #10999 for removing absolute paths from the generated container:
instead of trying to fix the container file after it has been dumped, telling to the PhpDumper where its output will be written allows it to replace parts of strings by an equivalent value based on `__DIR__`.
This should be safe, thus the PR is on 2.3.

Commits
-------

edd7057 [DependencyInjection] make paths relative to __DIR__ in the generated container
b604b0a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 27, 2014
  1. @fabpot
  2. @BenoitLeveque @fabpot

    Fix "absolute path" when we look to the cache directory

    BenoitLeveque committed with fabpot
    We should get __DIR__ instead of __DIR__ . '/.'
  3. @fabpot

    fixed CS

    fabpot committed
  4. @artursvonda @fabpot

    Make rootPath part of regex greedy

    artursvonda committed with fabpot
    - Fixes #10977
  5. @fabpot

    [HttpKernel] simplified some tests

    fabpot committed
View
60 src/Symfony/Component/HttpKernel/Kernel.php
@@ -32,6 +32,7 @@
use Symfony\Component\Config\Loader\DelegatingLoader;
use Symfony\Component\Config\ConfigCache;
use Symfony\Component\ClassLoader\ClassCollectionLoader;
+use Symfony\Component\Filesystem\Filesystem;
/**
* The Kernel is the heart of the Symfony system.
@@ -52,6 +53,7 @@
protected $bundleMap;
protected $container;
protected $rootDir;
+ protected $realRootDir;
protected $environment;
protected $debug;
protected $booted = false;
@@ -714,10 +716,68 @@ protected function dumpContainer(ConfigCache $cache, ContainerBuilder $container
$content = static::stripComments($content);
}
+ $content = $this->removeAbsolutePathsFromContainer($content);
+
$cache->write($content, $container->getResources());
}
/**
+ * Converts absolute paths to relative ones in the dumped container.
+ */
+ private function removeAbsolutePathsFromContainer($content)
+ {
+ if (!class_exists('Symfony\Component\Filesystem\Filesystem')) {
+ return $content;
+ }
+
+ $rootDir = $this->getRealRootDir();
+ if (!$rootDir) {
+ return $content;
+ }
+
+ $rootDir = rtrim($rootDir, '/');
+ $cacheDir = $this->getCacheDir();
+ $filesystem = new Filesystem();
+
+ return preg_replace_callback("{'([^']*?)(".preg_quote($rootDir)."[^']*)'}", function ($match) use ($filesystem, $cacheDir) {
+ $prefix = !empty($match[1]) ? "'$match[1]'.__DIR__" : "__DIR__";
+
+ if ('.' === $relativePath = rtrim($filesystem->makePathRelative($match[2], $cacheDir), '/')) {
+ return $prefix;
+ }
+
+ return $prefix.".'/".$relativePath."'";
+ }, $content);
+ }
+
+ /**
+ * Find the "real" root dir (by finding the composer.json file)
+ *
+ * @return null|string
+ */
+ private function getRealRootDir()
+ {
+ if (null !== $this->realRootDir) {
+ return $this->realRootDir;
+ }
+
+ $rootDir = $this->getRootDir();
+ $previous = $rootDir;
+ while (!file_exists($rootDir.'/composer.json')) {
+ if ($previous === $rootDir = realpath($rootDir.'/..')) {
+ // unable to detect the project root, give up
+ return $this->realRootDir = false;
+ }
+
+ $previous = $rootDir;
+ }
+
+ $this->realRootDir = $rootDir;
+
+ return $this->realRootDir;
+ }
+
+ /**
* Returns a loader for the container.
*
* @param ContainerInterface $container The service container
View
8 .../Component/HttpKernel/Tests/Fixtures/DumpedContainers/app/cache/dev/withAbsolutePaths.php
@@ -0,0 +1,8 @@
+'ROOT_DIR/app/cache/dev/foo'
+'ROOT_DIR/app/cache/foo'
+'ROOT_DIR/foo/bar.php'
+'ROOT_DIR/app/cache/dev'
+
+'/some/where/else/foo'
+
+'file:ROOT_DIR/app/cache/dev/profiler'
View
8 ...mponent/HttpKernel/Tests/Fixtures/DumpedContainers/app/cache/dev/withoutAbsolutePaths.php
@@ -0,0 +1,8 @@
+__DIR__.'/foo'
+__DIR__.'/../foo'
+__DIR__.'/../../../foo/bar.php'
+__DIR__
+
+'/some/where/else/foo'
+
+'file:'.__DIR__.'/profiler'
View
1 src/Symfony/Component/HttpKernel/Tests/Fixtures/DumpedContainers/composer.json
@@ -0,0 +1 @@
+{}
View
10 src/Symfony/Component/HttpKernel/Tests/Fixtures/KernelForTest.php
@@ -16,6 +16,11 @@
class KernelForTest extends Kernel
{
+ public function setRootDir($dir)
+ {
+ $this->rootDir = $dir;
+ }
+
public function getBundleMap()
{
return $this->bundleMap;
@@ -34,4 +39,9 @@ public function isBooted()
{
return $this->booted;
}
+
+ public function setRealRootDir($dir)
+ {
+ $this->realRootDir = $dir;
+ }
}
View
33 src/Symfony/Component/HttpKernel/Tests/KernelTest.php
@@ -723,6 +723,39 @@ public function testTerminateDelegatesTerminationOnlyForTerminableInterface()
}
/**
+ * @dataProvider getRemoveAbsolutePathsFromContainerData
+ */
+ public function testRemoveAbsolutePathsFromContainer($symfonyRootDir, $realRootDir, $replacement, $replaced)
+ {
+ $kernel = new KernelForTest('dev', true);
+ $kernel->setRootDir($symfonyRootDir);
+ if (null !== $realRootDir) {
+ $kernel->setRealRootDir($realRootDir);
+ }
+
+ $content = file_get_contents(__DIR__.'/Fixtures/DumpedContainers/app/cache/dev/withAbsolutePaths.php');
+ $content = str_replace('ROOT_DIR', $replacement, $content);
+
+ $m = new \ReflectionMethod($kernel, 'removeAbsolutePathsFromContainer');
+ $m->setAccessible(true);
+ $newContent = $m->invoke($kernel, $content);
+ if ($replaced) {
+ $this->assertEquals(file_get_contents(__DIR__.'/Fixtures/DumpedContainers/app/cache/dev/withoutAbsolutePaths.php'), $newContent);
+ } else {
+ $this->assertEquals($newContent, $content);
+ }
+ }
+
+ public function getRemoveAbsolutePathsFromContainerData()
+ {
+ return array(
+ array(__DIR__.'/Fixtures/DumpedContainers/app', null, __DIR__.'/Fixtures/DumpedContainers', true),
+ array(sys_get_temp_dir(), null, __DIR__.'/Fixtures/DumpedContainers', false),
+ array('/app/app', '/app', '/app', true),
+ );
+ }
+
+ /**
* Returns a mock for the BundleInterface
*
* @return BundleInterface
View
4 src/Symfony/Component/HttpKernel/composer.json
@@ -29,6 +29,7 @@
"symfony/console": "~2.2",
"symfony/dependency-injection": "~2.0",
"symfony/finder": "~2.0",
+ "symfony/filesystem": "~2.4",
"symfony/process": "~2.0",
"symfony/routing": "~2.2",
"symfony/stopwatch": "~2.2",
@@ -40,7 +41,8 @@
"symfony/config": "",
"symfony/console": "",
"symfony/dependency-injection": "",
- "symfony/finder": ""
+ "symfony/finder": "",
+ "symfony/filesystem": ""
},
"autoload": {
"psr-0": { "Symfony\\Component\\HttpKernel\\": "" }
Something went wrong with that request. Please try again.