Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handshake environment cannot be properly integrated with CageFS #2145

Open
abashurov opened this issue Dec 5, 2018 · 2 comments

Comments

@abashurov
Copy link

commented Dec 5, 2018

Issue report

Question 1: What is the problem?

Problem description
The application startup fails if the user is placed behind CageFS

Steps to reproduce:

  1. Deploy Passenger on the CloudLinux platform
  2. Configure a vhost with a separate user, this user should be located behind CageFS:
    cagefsctl --init
    cagefsctl --enable-all
  3. Create a sample application in the vhost and enable Node.js processing for it:
    const http = require('http')
    http.createServer((q, a) => { 
        a.writeHead(200, {'Content-Type': 'text/plain'})
        a.end("Hello, World!\n")
    }).listen(process.env.PORT)
    

Actual result
Passenger is unable to complete the handshake and stops the attempts to spawn the application

Expected result
The handshake is performed and the application pool is successfully initiated

Failure logs:

App 49406 output: Warning: cannot open /tmp/passenger.spawn.XXXX9gDaFR/envdump/user_info for writing
App 49406 output: Warning: cannot open /tmp/passenger.spawn.XXXX9gDaFR/envdump/envvars for writing
App 49406 output: Warning: cannot open /tmp/passenger.spawn.XXXX9gDaFR/envdump/user_info for writing
App 49406 output: Warning: cannot open /tmp/passenger.spawn.XXXX9gDaFR/envdump/envvars for writing
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/execute_through_os_shell: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_before_shell/state: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_before_shell/begin_time_monotonic: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_before_shell/end_time_monotonic: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_os_shell/state: No such file ordirectory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_os_shell/begin_time_monotonic: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_after_shell/state: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_after_shell/begin_time_monotonic: No such file or directory (errno=2)
App 49406 output: Error: Cannot open '/tmp/passenger.spawn.XXXX9gDaFR/args.json' for reading: No such file or directory (errno=2)
App 49406 output:      (empty)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_after_shell/state: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_after_shell/begin_time_monotonic: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/steps/subprocess_spawn_env_setupper_after_shell/end_time_monotonic: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/error/category: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/error/summary: No such file or directory (errno=2)
App 49406 output: Warning: Cannot create file /tmp/passenger.spawn.XXXX9gDaFR/response/error/advanced_problem_details: No such file or directory (errno=2)
[ E 2018-12-05 08:43:16.1067 49302/Tc age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /var/www/vhosts/example.com/httpdocs: The application process exited prematurely.
  Error ID: 8f4f225d
  Error details saved to: /tmp/passenger-error-rby3kv.html

Possible workaround
Since HandshakeWorkDir does not accept any configuration options, the TMPDIR can be modified for the whole web-server to enable pass-through in the CageFP mountpoint:

  1. systemctl edit httpd
    [Service]
    Environment=TMPDIR=/var/passenger
    
  2. Create a directory to hold the handshake session and set the proper permissions:
    mkdir /var/passenger
    chmod 1777 /var/passenger
  3. Add the directory to the CageFS mountpoints, to allow users to access it:
    echo '/var/passenger/' >> /etc/cagefs/cagefs.mp
  4. Propagate the changes to all CageFS users:
    cagefsctl --remount-all

Note: The workaround is not a viable one, as it exposes webserver's temp directory to all users

Question 2: Passenger version and integration mode:

OSS Passenger 5.3.5 Apache/nginx module, OSS Passenger 6.0.0 Apache module

Question 3: OS or Linux distro, platform (including version):

CloudLinux 7.5 x86_64

Question 4: Passenger installation method:

[x] Phusion YUM repo
[x] Other, please specify:
Passenger packages built by Plesk

Question 5: Your app's programming language (including any version managers) and framework (including versions):

Node.js 9.x

Question 6: Are you using a PaaS and/or containerization? If so which one?

No containerization is used.

Question 7: Anything else about your setup that we should know?

System users are separated though the CloudLinux CageFS, which gives them private /tmp.

@CamJN

This comment has been minimized.

Copy link
Contributor

commented Dec 11, 2018

I agree that this would be better if it was configurable. Not knowing much about cageFS, is it necessary to allow all CageFS users to access the directory? Wouldn't it be fine to limit it to the user running apache/nginx/passenger?

@abashurov

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Unfortunately, I am also not an expert on the matter. From what I see, the handshake fails due to the fact that the directory is spawned on the host system, outside of jail, and the connection to it is performed by the module after it enters the LVE.
So the only way to properly pass-through the directory from the host to the jail is seems to be to create an appropriate CageFS mount-point, which in this case will host the whole $TMPDIR.

Here are the full STR, that should be enough to reproduce the issue, given that CloudLinux still offers trial licenses:

  1. Create a CentOS 7 VM, convert it to CloudLinux with a trial license and reboot
  2. Make sure that the LVE kernel is active and cagefsctl is present:
    # uname -r
    3.10.0-714.10.2.lve1.5.17.el7.x86_64
    # cagefsctl --version
    6.1-27.el7.cloudlinux
    
  3. Init the CageFS:
    cagefsctl --init
    
  4. Add the Passenger repo and install the OSS Passenger 6.0
  5. Install the Node.js for further use
  6. Create a vhost with user and configure it to use Passenger with a Node.js application:
    useradd sampleuser
    chgrp apache /home/sampleuser/
    chmod g+x /home/sampleuser
    mkdir /etc/httpd/conf/vhosts
    echo 'Include /etc/httpd/conf/vhosts/*.conf' >> /etc/httpd/conf/httpd.conf
    systemctl start httpd
    
    # cat /etc/httpd/conf/vhosts/example.com.conf
    <VirtualHost *:80>
            ServerName "example.com:80"
            ServerAlias "www.example.com"
            DocumentRoot "/home/sampleuser/"
            <Directory "/home/sampleuser/">
                    Require all granted
                    PassengerEnabled on
                    PassengerAppType node
                    PassengerNodejs "/usr/bin/node"
                    PassengerUser "sampleuser"
                    PassengerGroup "sampleuser"
                    PassengerAppRoot "/home/sampleuser/"
                    PassengerStartupFile "app.js"
                    PassengerAppEnv "production"
            </Directory>
    </VirtualHost>
    
  7. Add Passenger and InstReg directory to the CageFS jail template and hide the user behind CageFS:
    cagefsctl --enable-cagefs
    cagefsctl --enable sampleuser
    echo '/var/run/passenger-instreg/' >> /etc/cagefs/cagefs.mp
    cagefsctl --addrpm passenger
    cagefsctl --force-update
    cagefsctl --remount-all
    
  8. Create the /home/sampleuser/app.js file:
    const http = require('http')
    http.createServer((q, a) => { 
        a.writeHead(200, {'Content-Type': 'text/plain'})
        a.end("Hello, World!\n")
    }).listen(process.env.PORT)
    
  9. Restart Apache
plmnikulin added a commit to plmnikulin/passenger that referenced this issue Apr 18, 2019
Add Apache

    InstanceRegistryDir /run/passenger/registry

and Nginx

    passenger_spawn_dir /run/passenger/spawn;

configuration options to set spawn directory used during handshake.
Hardcoded /tmp is unacceptable for CloudLinux CageFS. Full support
would require per-user directories for handshake and registry.
Current implementation is a kind of trade-off, it does not allow
real illusion of the only user on the server.
plmnikulin added a commit to plmnikulin/passenger that referenced this issue Apr 18, 2019
CamJN added a commit that referenced this issue May 9, 2019
* Configurable spawn dir for CageFS (GH-2145)

Add Apache

    InstanceRegistryDir /run/passenger/registry

and Nginx

    passenger_spawn_dir /run/passenger/spawn;

configuration options to set spawn directory used during handshake.
Hardcoded /tmp is unacceptable for CloudLinux CageFS. Full support
would require per-user directories for handshake and registry.
Current implementation is a kind of trade-off, it does not allow
real illusion of the only user on the server.

* Configurable spawn dir - codegen files (GH-2145)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.