Bug: apache mod_userdir causes weird URL segment duplication

[evansharp]

Describe the bug
When using apache's mod_userdir to enable each server user to have a public site, CI's request class chokes on either the url rewriting or the tilde(~) in the url, making all controllers unreachable.

When the url string is echoed as a breakpoint at the top of the Config\Routes.php (with echo (string)$request->uri; ), the userdir segment and install directory are both repeated, causing the router to always interpret the wrong segments of the URL.

i.e. http://domain.com/~username/subdir/ becomes http://domain.com/~username/subdir/~username/subdir/ in the Request.

CI is bootstrapping fine and all system paths are resolving. Requesting the baseUrl though, that should be the default controller (splash screen), returns the CI 404 page citing a controller not found "\App\Controllers\ ~username::subdir".

My attempt to get support in the forums first is here. I can pastebin the rewrite log if it could be useful.

CodeIgniter 4 version
Whatever version is the dependancy for version 4.1.1 of the starter app on Packagist. Looks like 4.1.1-dev?

Affected module(s)
Probably HTTP/Request but might be Router?

Expected behavior, and steps to reproduce if appropriate
I expect the default controller to be served when requesting the baseUrl.

Reproduce on a system running mod_userdir and put CI in a subdirectory of the user's ~/public_html.

I have tried

• Setting the RewriteBase directive in public/.htaccess to /~userdir/subdir/; the issue does not seem to be with Apache's rewriting.
• All combinations of setting the baseUrl in .env and Config\App to include the segments
• Doing dd( $request->uri ) is a bit baffling. CI seems to have the base url, but finds the last two as additional segments anyway.

Context

• OS: Ubuntu 20.04
• Web server: Apache 2.4
• PHP version: 7.4

[evansharp]

I continued to investigate this by probing the Request object at the top of the Routes.php config.

I discovered that the path portion of the Request contained the problematic URI segments and if I used $request->uri->setPath(''), I could get away from the 404 error I had.

However, adjusting the path this way is doing something else, since instead I get ReflectionException after something propagates through the debug toolbar? The Home (default appstarter) controller is 'not found' even though the file is listed in the backtrace file list.

I am going to try move the path edit into a 'before' filter to see if I can head off this exception.

[evansharp]

... setting the IncomingRequest path to null manually in a global before filter did not prevent the duplicated segments appearing at my \Config\Routes.php breakpoint.

[evansharp]

I have found a workaround:

If I prefix every route definition with the mod_userdir and install directory segments, then they will match properly and route to the controller.

I still still think there is a bug somewhere here with respect to mod_userdir and mod_rewrite, but I'm at the limit of what I can reverse-engineer about how CI handles the path portion of the Request. I do hope this ticket gets looked at for a proper config or mod_rewrite-based patch.

[MGatner]

Thanks for the report. I’ve never used mod_userdir but a cursory glance at it makes me think it should not affect CI4 of your Apache Config and .htaccess are configured accordingly. My first guess on the “doubling” is that your baseUrl is not set correctly relative to the Apache directives. Try adjusting that in Config\App and if you are still having problems see if you can recreate the issue in a barebones AppStarter repo so it is easy to reproduce.

[evansharp]

Thanks for this @MGatner!

I have verified my baseUrl many times and in many ways. I'm currently using the .env, but I've tried different values in \Config\App.php too. If there was something wrong with that config item, why would pre-pending my routes fix it? This seems like a pattern-matching issue in the comparison of the request with the baseUrl variable.

Are we sure the regex that parses that string is accepting the tilde (~) as intended?

The installation being assessed here is a brand new barebones AppStarter clone. It's why I posted the forum thread in "Installation and Config" rather than general Support.

Any other ideas?
Thanks!

[MGatner]

My hunch is the issue resides in a layer before the PHP of the framework (like .htaccess) - but I will need to read more on mod_userdir before I can be of any real help in that area. To be continued once I am back on desktop...

[evansharp]

@MGatner Here is a scrubbed paste of the trace6 log I generated: https://pastebin.com/ZUnnv1C8

This shows everything apache does for mod_rewrite from the start of the request to the final serve of ci4/public/index.php.

www in this case is not /var/www/, it is /home/username/www/, which is the user's home directory sub I've configured to be served by mod_userdir. The default is public_html, but I found that too close to CI's pathing for clarity.

~/www/ has no .htaccess. Here is ~/www/ci4/.htaccess:

<IfModule mod_rewrite.c>
    RewriteEngine on
    # Redirect requests to public
    RewriteBase /~username/ci4/                  <--- have tried with and without this, doesn't seem to matter
    RewriteRule  ^$ public/    [L]
    RewriteRule  (.*) public/$1 [L]
</IfModule>

Here is ~/www/username/ci4/public/.htaccess:

# Disable directory browsing
Options All -Indexes

# ----------------------------------------------------------------------
# Rewrite engine
# ----------------------------------------------------------------------

# Turning on the rewrite engine is necessary for the following rules and features.
# FollowSymLinks must be enabled for this to work.
<IfModule mod_rewrite.c>
        Options +FollowSymlinks
        RewriteEngine On

        # If you installed CodeIgniter in a subfolder, you will need to
        # change the following line to match the subfolder you need.
        # http://httpd.apache.org/docs/current/mod/mod_rewrite.html#rewritebase
         RewriteBase /~username/ci4/                         <--- have tried with and without this, doesn't seem to matter

        # Redirect Trailing Slashes...
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteCond %{REQUEST_URI} (.+)/$
      RewriteRule ^ %1 [L,R=301]

        # Rewrite "www.example.com -> example.com"
        RewriteCond %{HTTPS} !=on
        RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
        RewriteRule ^ http://%1%{REQUEST_URI} [R=301,L]

        # Checks to see if the user is attempting to access a valid file,
    # such as an image or css document, if this isn't true it sends the
    # request to the front controller, index.php
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteRule ^([\s\S]*)$ index.php/$1 [L,NC,QSA]

        # Ensure Authorization header is passed along
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
</IfModule>

<IfModule !mod_rewrite.c>
    # If we don't have mod_rewrite installed, all 404's
    # can be sent to index.php, and everything works as normal.
    ErrorDocument 404 index.php
</IfModule>

# Disable server signature start
    ServerSignature Off
# Disable server signature end

My pretty standard userdir.conf:

<IfModule mod_userdir.c>
        UserDir www
        UserDir disabled root ubuntu

        <Directory /home/*/www>
                AllowOverride All
                Options MultiViews Indexes FollowSymLinks
                <Limit GET POST OPTIONS>
                        Require all granted
                </Limit>
                <LimitExcept GET POST OPTIONS>
                        Require all denied
                </LimitExcept>
        </Directory>
</IfModule>

[John-Betong]

I have tried to install a CI4 application that uses your directory structure and was also not able to call the Controller from a public sub-directory?

Here is a rough online demo called from the public directory with all the ~userdirs above the root.

https://this-is-a-test-to-see-if-it-works.tk/~evan

@evansharp,
Is there any reason the above setup cannot be used instead of mod_userdir(...)?

[MGatner]

I can't speak much to the Apache layer of things but let's figure out what is reaching CodeIgniter to make sure everything coming "in" is correct. Can you interrupt the boot process (try app/Config/Boot/ with a dd($_SERVER) and share the relevant parts? Be mindful that CI4 injects some passwords into that array which you will not want to share here.

[John-Betong]

@MGatner,
$_SERVER details are now being displayed.

Please note that the online demo appears to be working correctly.

The problem is not being able to call CI4 from a public/evansharp sub-directory. @evansharp had the same problem with not being able to call an app/Controller/C_controllername.php which seems more than a coincidence that we both failed to call the necessary controller.

I could zip the app folder if you want?

[John-Betong]

Cracked it :)

Modified app/Config/Paths.php

define('TMP', '/var/www/this-is-a-test-to-see-if-it-works.tk/');
define('VIEWPATH', TMP .'evansharp/Views');

class Paths
{
Modified app/Config/Paths -> public $viewDirectory   = TMP .'evansharp/Views';
}

This should now work with mod_userdir(...);

https://this-is-a-test-to-see-if-it-works.tk/

[John-Betong]

Please accept my apologies because although changing the app/Config/Paths.php should have been a solution. Further investigation discovered that setting $viewDirectory has absolutely no effect? It appears as though the app/Config/Views/ directory is used regardless?

Sorry :(

VIEWPATH is also not defined anywhere unlike CI3?

File: app/Config/Paths.php

/** * --------------------------------------------------------------- * VIEW DIRECTORY NAME * --------------------------------------------------------------- * * This variable must contain the name of the directory that * contains the view files used by your application. By * default this is inapp/Views. This value * is used when no value is provided to Services::renderer()`.
*
* @var string
*/
# WRONG DIRECTORY HAS NO EFFECT AND IS IGNORED
public $viewDirectory = 'NOT_USED' .'evansharp/Views';
public $viewPath = 'ALSO NOT USED - FCPATH' .'evansharp/';

`

[John-Betong]

https://forum.codeigniter.com/thread-78934.html
Many, many thanks to @bobkendrick@blueyonder.co.uk regardin g the above Thread.

I modified the $viewPath and added "Views" 👍
$viewPath = '[viewDirectory] => /var/www/this-is-a-test-to-see-if-it-works.tk/public_html/evansharpViews/';

Also added this KLUDGE:
file: .system/Views/View.php => line: 192
` $this->renderVars['file'] = $this->viewPath . $this->renderVars['view'];

# KLUDGE: 2021-03-30 Start	
 	if(strpos( $this->renderVars['file'], 'Views/evansharpViews/') ):
 			$bad  = 'evansharpViews/evansharpViews/';
 			$good = 'evansharpViews/';
 			$this->renderVars['file'] = str_replace($bad, $good, $this->renderVars['file']);
 	endif;
# KLUDGE: 2021-03-30 FINISH

	if (! is_file($this->renderVars['file']))
	{
		$this->renderVars['file'] = $this->loader->locateFile($this->renderVars['view'], 'Views', empty($fileExt) ? 'php' : $fileExt);

}
`
This now works with the new Views Path below the public/public_html path
https://this-is-a-test-to-see-if-it-works.tk/

Just noticed that an incorrect URL fails :)

[John-Betong]

The KLUDGE was bothering me so I decided to investigate and discovered the following changes removed the need for the KLUDGE:

file: .app/Conftollers/C_Others.php
`<?php declare(strict_types=1);

namespace App\Controllers;

// =============================
class C_Others extends BaseController
{

// =============================
public function index( string $other='' ) : string
{
return view('v_evansharp', $this->data);
}//

// =============================
public function others( string $other='' ) : string
{
$this->data['BAD_URL'] = ''; // DEFAULT empty otherwise view('v_BAD_URL', ...);

if( empty($other) ) :
# return view('v_evansharp', $this->data);

elseif( in_array($other, $this->data['pages']) ) :
# return view('evansharpViews/v_evansharp', $this->data);

else: # DOES NOT EXIST
$this->data['BAD_URL'] = view('v_BAD_URL', $this->data, [TRUE]);

  # return view('evansharpViews/v_evansharp', $this->data);

endif;

return view('v_evansharp', $this->data);
}//

}///
`
Also fixed the incorrect URL and added a test for BAD_URL.

[evansharp]

Impressive detective work @John-Betong! I find it hard to follow in comments, will you PR the patch?

Ultimately does correcting this viewPath hardcoding allow the router to follow mod_userdir rewrites properly?

[evansharp]

Bump? @John-Betong @MGatner ?

[John-Betong]

[quote]
@evansharp
Impressive detective work @John-Betong! I find it hard to follow in comments, will you PR the patch?
[/quote]
As mentioned CI3 created a VIEWPATH constant that allowed the ./app/Views/ directory to be placed outside of the ./app/ path. For reasons that I do not know this feature has been dropped from CI4.

[quote]
Ultimately does correcting this viewPath hardcoding allow the router to follow mod_userdir rewrites properly?
[/quote]
Yes it works fine as can be seen in the online demo:
https://this-is-a-test-to-see-if-it-works.tk/~evan

Please note that CI_VERSION 4.1.1 is used and PHP 8.0.5.
I think it best if a Pull Request is not raised since this changing the default ./app/Views directory is a user specific modification.

Perhaps @MGatner has different thoughts?

[lonnieezell]

As mentioned CI3 created a VIEWPATH constant that allowed the ./app/Views/ directory to be placed outside of the ./app/ path. For reasons that I do not know this feature has been dropped from CI4.

When you grab a new renderer service, you can specify the viewpath you want to use there. The one caveat is that the default one in the system already has the standard viewpath set by the time it gets to the controller but you can get around that by grabbing a non-shared instance and saving it in a base controller or similar:

$this->renderer = service('renderer', ROOTPATH .'Views/');

Additionally remember you can use namespaces to access views so you can define a namespace that points to where you want your views to hang out, and then call them with the namespaced version of the view name.

[evansharp]

Forgive me @John-Betong and @MGatner, but we lost the plot around

Cracked it :)

Modified app/Config/Paths.php

define('TMP', '/var/www/this-is-a-test-to-see-if-it-works.tk/');
define('VIEWPATH', TMP .'evansharp/Views');

class Paths
{
Modified app/Config/Paths -> public $viewDirectory   = TMP .'evansharp/Views';
}

This should now work with mod_userdir(...);

https://this-is-a-test-to-see-if-it-works.tk/

This directed the investigation towards the viewpath, when the issue remains having the router locate a controller and method first.

Take a look at my student's project: https://code.coastmountainacademy.ca/~kaimartin/hiperplanner/
He's trying to implement a simple auth system with a filter and two controllers. As a workaround that was working for a while, I had him prepend routes with the those URL segments being mis-interpreted:
file: .app/Config/Routes.php

// Prefix workaround for mod_userdir issue
// Make ALL your routes start with the "path" by pre-pending
// the route pattern with this variable:
$path = '/~kaimartin/hiperplanner';

// defualt route:
$routes->get($path, 'Pages::index');

$routes->get($path . '/login', 'Login::index');
$routes->get($path . '/login/auth', 'Login::auth');
$routes->get($path . '/addentry', 'Addentry::index');

$routes->get($path . '/register', 'Register::index');
$routes->get($path . '/register/save', 'Register::save');

$routes->get($path . '/dashboard', 'Dashboard::index',['filter' => 'auth']);

We've hit the speedbump again though when the registration form gets submitted and tries to access [...]/register/save.
If I directly access the full URL [...]/register/save, I can get to the controller (https://snipboard.io/oqYDXR.jpg).
When the form submits (the action is also the FULL URL) I get the controller not found (https://snipboard.io/NvnS2w.jpg).

You both know the boot better than I do, are there other suggestions than a pathing issue?

[evansharp]

I think I've got it, but it's a bummer:

Apache mod_userdir follows a configuration directive in /etc/apache2/mods-available/userdir.conf to determine the directory in each user's home/ to serve when www.domain/~user is requested. By default this is ~/html_public. It must be the same for all users.

I now realize that simply placing a whole CI install (./app,./vendor, ./writable, ./public, etc.) into ~/html_public isn't going to work because the front controller ./public/index.php needs to be served by apache for all the other path bootstrapping to work properly. This is what has not been happening; my route workaround was just hacking the path bootstrap.

Either mod_userdir must be configured to go deeper and serve ~/html_public/public OR the CI install needs to be placed right in the user's home and mod_userdir configured to serve CI's ./public.

Neither option is very clean since not every user of a shared host will be using it for CI, nor want to have the CI install in their home/. I am curious if anyone has an idea for redirecting apache from the install route into ./public in such a way that the subsequent url rewrites are not negatively affected; I think this will be the only way for a clean and universal support for mod_userdir.

[MGatner]

There are lots of threads on the forums about serving and securing root-directory index files, since it is a frequent issue with shared hosting.

Apache directives have plenty of conditionals - could you not say "if exists ~/html_public/public, otherwise ~/public"?

[evansharp]

Apache directives have plenty of conditionals - could you not say "if exists ~/html_public/public, otherwise ~/public"?

Yes, this is possible, but requires a nuanced server configuration. The third 'additional example' on the main docs page demonstrates a directive for multiple failover targets per user.

This seems like useful info for the userguide; I'm going to draft an addition to the 'Running your app/Hosting with Apache' section. Not that we want to get into documenting Apache and Ngix, but explaining this logic could save some pain.

I'm going to close this issue since enfin it is clearly not a CI bug.