Permission denied (spawn /tmp/ EACCES)

[jcfbeardsley]

Summary

Hi team,

I have a new Cronicle installation with one master machine and two remotes. I have used Cronicle extensively on other machines (older releases) and often need to run jobs as other (non-root) users, using a copy of the Shell Script plugin with an appropriate UID set. This has worked well for me in the past, but on the current release I'm seeing this error being thrown:

Permission denied (spawn /tmp/cronicle-script-temp-jlomeq1t60u.sh EACCES)

When trying to execute any shell script as a non-root user. Both Cronicle (/opt/cronicle/) and node (/usr/bin/node) have appropriate user permissions and read/write access to the /tmp dir (can touch new files to /tmp). Running the same commands manually from the user in a shell session also work as expected.

Am I missing something obvious here? In setting up this server, I did import a Cronicle export (jobs and plugins) from an older release of Cronicle (v0.8.56), which potentially has replaced the default Shell Plugin with an older version and created issues? I don't believe this is related to the multi-server setup (the same result occurs on the master server for a simple echo test). All commands run fine using the Shell Script plugin as root.

Thoughts?

Steps to reproduce the problem

• Installed Cronicle to /opt/cronicle as root.
• Imported Cronicle backup from an older release of Cronicle (v0.8.56) - includes jobs, schedule and plugins.
• Copied Shell Script plugin and added a UID to run as a different user.
• Called the plugin and ran a job (simple echo test), triggers a job failure.

Your Setup

Operating system and version?

Ubuntu 22.04

Node.js version?

v20.9.0 (LTS)

Cronicle software version?

v0.9.38

Are you using a multi-server setup, or just a single server?

Multi-server, but job failure happens if job is run on the master server too.

Are you using the filesystem as back-end storage, or S3/Couchbase?

Filesystem

Can you reproduce the crash consistently?

Yes

Log Excerpts

[1699245207.977][2023-11-06 15:33:27][cronicle-master][1049572][Cronicle][debug][6][Running event manually: Test Event][{"enabled":1,"params":{"script":"#!/bin/sh\n\n# Enter your shell script code here","annotate":0,"json":0},"timing":false,"max_children":1,"timeout":3600,"catch_up":0,"queue_max":1000,"timezone":"Australia/Hobart","plugin":"pjjxktka801","title":"Test Event","category":"ckaogaswg35","target":"cronicle-master","algo":"random","multiplex":0,"stagger":0,"retries":0,"retry_delay":0,"detached":0,"queue":0,"chain":"","chain_error":"","notify_success":"","notify_fail":"","web_hook":"","cpu_limit":0,"cpu_sustain":0,"memory_limit":0,"memory_sustain":0,"log_max_size":0,"notes":"","id":"elomei4ha0s","modified":1699244838,"created":1699244838,"username":"admin","now":1699244843,"source":"Manual (admin)"}]
[1699245207.978][2023-11-06 15:33:27][cronicle-master][1049572][Cronicle][debug][6][Launching local job][{"params":{"script":"#!/bin/sh\n\n# Enter your shell script code here","annotate":0,"json":0},"timeout":3600,"catch_up":0,"queue_max":1000,"timezone":"Australia/Hobart","plugin":"pjjxktka801","category":"ckaogaswg35","target":"cronicle-master","algo":"random","multiplex":0,"stagger":0,"retries":0,"retry_delay":0,"detached":0,"queue":0,"chain":"","chain_error":"","notify_success":"","notify_fail":"","web_hook":"","cpu_limit":0,"cpu_sustain":0,"memory_limit":0,"memory_sustain":0,"log_max_size":0,"notes":"","now":1699244843,"source":"Manual (admin)","id":"jlomeq1t60u","time_start":1699245207.978,"hostname":"cronicle-master","event":"elomei4ha0s","event_title":"Test Event","plugin_title":"Shell Script (cronicle)","category_title":"RSync/RClone","nice_target":"cronicle-master","command":"/opt/cronicle/bin/shell-plugin.js","cwd":"/home/cronicle/","uid":674639,"log_file":"/opt/cronicle/logs/jobs/jlomeq1t60u.log"}]
[1699245207.978][2023-11-06 15:33:27][cronicle-master][1049572][Cronicle][debug][9][Child spawn options:][{"cwd":"/home/cronicle/","uid":674639,"gid":674639,"env":{"SUDO_GID":"711651","LESSOPEN":"| /usr/bin/lesspipe %s","MAIL":"/var/mail/root","USER":"cronicle","SHLVL":"1","HOME":"/home/cronicle","OLDPWD":"/opt/cronicle/bin","NODE_TLS_REJECT_UNAUTHORIZED":"0","SUDO_UID":"711651","LOGNAME":"root","_":"./control.sh","TERM":"xterm-256color","PATH":"/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/bin:/usr/sbin:/sbin:/usr/local/sbin","LANG":"en_AU.UTF-8","LS_COLORS":"rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:","SUDO_COMMAND":"/usr/bin/su","SHELL":"/bin/bash","KRB5CCNAME":"FILE:/tmp/krb5cc_711651_0jH6CL","SUDO_USER":"bea195","LESSCLOSE":"/usr/bin/lesspipe %s %s","PWD":"/home/cronicle/","__daemon":"true","CRONICLE":"0.9.38","JOB_ID":"jlomeq1t60u","JOB_LOG":"/opt/cronicle/logs/jobs/jlomeq1t60u.log","JOB_NOW":"1699244843","JOB_TIMEOUT":"3600","JOB_CATCH_UP":"0","JOB_QUEUE_MAX":"1000","JOB_TIMEZONE":"Australia/Hobart","JOB_PLUGIN":"pjjxktka801","JOB_CATEGORY":"ckaogaswg35","JOB_TARGET":"cronicle-master","JOB_ALGO":"random","JOB_MULTIPLEX":"0","JOB_STAGGER":"0","JOB_RETRIES":"0","JOB_RETRY_DELAY":"0","JOB_DETACHED":"0","JOB_QUEUE":"0","JOB_CHAIN":"","JOB_CHAIN_ERROR":"","JOB_NOTIFY_SUCCESS":"","JOB_NOTIFY_FAIL":"","JOB_WEB_HOOK":"","JOB_CPU_LIMIT":"0","JOB_CPU_SUSTAIN":"0","JOB_MEMORY_LIMIT":"0","JOB_MEMORY_SUSTAIN":"0","JOB_LOG_MAX_SIZE":"0","JOB_NOTES":"","JOB_SOURCE":"Manual (admin)","JOB_TIME_START":"1699245207.978","JOB_HOSTNAME":"cronicle-master","JOB_EVENT":"elomei4ha0s","JOB_EVENT_TITLE":"Test Event","JOB_PLUGIN_TITLE":"Shell Script (cronicle)","JOB_CATEGORY_TITLE":"RSync/RClone","JOB_NICE_TARGET":"cronicle-master","JOB_COMMAND":"/opt/cronicle/bin/shell-plugin.js","JOB_CWD":"/home/cronicle/","JOB_UID":"674639","JOB_LOG_FILE":"/opt/cronicle/logs/jobs/jlomeq1t60u.log","USERNAME":"cronicle","SCRIPT":"#!/bin/sh\n\n# Enter your shell script code here","ANNOTATE":"0","JSON":"0"}}]
[1699245207.98][2023-11-06 15:33:27][cronicle-master][1049572][Cronicle][debug][3][Spawned child process: 1069321 for job: jlomeq1t60u][/opt/cronicle/bin/shell-plugin.js]
[1699245208.035][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][3][Child 1069321 exited with code: 0][]
[1699245208.035][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][5][Job completed with error][{"params":{"script":"#!/bin/sh\n\n# Enter your shell script code here","annotate":0,"json":0},"timeout":3600,"catch_up":0,"queue_max":1000,"timezone":"Australia/Hobart","plugin":"pjjxktka801","category":"ckaogaswg35","target":"cronicle-master","algo":"random","multiplex":0,"stagger":0,"retries":0,"retry_delay":0,"detached":0,"queue":0,"chain":"","chain_error":"","notify_success":"","notify_fail":"","web_hook":"","cpu_limit":0,"cpu_sustain":0,"memory_limit":0,"memory_sustain":0,"log_max_size":0,"notes":"","now":1699244843,"source":"Manual (admin)","id":"jlomeq1t60u","time_start":1699245207.978,"hostname":"cronicle-master","event":"elomei4ha0s","event_title":"Test Event","plugin_title":"Shell Script (cronicle)","category_title":"RSync/RClone","nice_target":"cronicle-master","command":"/opt/cronicle/bin/shell-plugin.js","cwd":"/home/cronicle/","uid":674639,"log_file":"/opt/cronicle/logs/jobs/jlomeq1t60u.log","pid":1069321,"complete":1,"code":1,"description":"Script failed: Permission denied (spawn /tmp/cronicle-script-temp-jlomeq1t60u.sh EACCES)"}]
[1699245208.035][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][6][Storing job log: /opt/cronicle/logs/jobs/jlomeq1t60u.log: jobs/jlomeq1t60u/log.txt.gz][]
[1699245208.042][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][9][Job log stored successfully: jobs/jlomeq1t60u/log.txt.gz][]
[1699245208.042][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][9][Deleting local file: /opt/cronicle/logs/jobs/jlomeq1t60u.log][]
[1699245208.042][2023-11-06 15:33:28][cronicle-master][1049572][Cronicle][debug][9][Successfully deleted local job log file: /opt/cronicle/logs/jobs/jlomeq1t60u.log][]

[jhuckaby]

It is likely that your server has been "hardened" so that scripts cannot be executed by non-root users in the temp directory (/tmp).

See issue #133 for possible workarounds.

Also, ChatGPT tells me that your /tmp filesystem may be mounted with the noexec option, and you could try to change /etc/fstab and remount, to remove that restriction.

[mcfetz]

If I remember correctly, I once had the same problem on my Synology NAS. Here it helped to change the TMP directory in the environment to a directory with Exec rights before starting Cronicle. That all went quite smoothly.

[mcfetz]

My two lines for starting Cronicle

export TMP=/opt/cronicle/tmp
/opt/cronicle/bin/control.sh start

[jcfbeardsley]

@jhuckaby @mcfetz Thanks for the prompt reply. This was indeed a security issue on our new corporate VMs applying to the /tmp directory. I've worked around this by setting the TMP var as above.