Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cbsd jexec jname="*" date .... hangs. #440

Closed
tm-up opened this issue Nov 2, 2019 · 5 comments
Closed

cbsd jexec jname="*" date .... hangs. #440

tm-up opened this issue Nov 2, 2019 · 5 comments
Assignees

Comments

@tm-up
Copy link

tm-up commented Nov 2, 2019

Running above command will hang (all related pkg commands too).
I see two related processes, while the command hangs:

/usr/local/bin/sudo -E /usr/local/cbsd/sudoexec/jexec jname=* date
/usr/local/bin/cbsd /usr/local/cbsd/sudoexec/jexec jname=* date

Both processes are marked stopped ('T' in ps -aux).
Output on console:

# cbsd jexec jname='*`
Hint: Press 'Ctrl+t' to see last logfile line for active task
 * [jexec: pending: 2:[ myjail testjail], active: 0]

Running the command cbsd jexec jname=myjail date works.

FreeBSD version: 12.0 RELEASE p11
CBSD version: 12.1.1

@tm-up
Copy link
Author

tm-up commented Nov 2, 2019

running that command with env CBSD_DEBUG=1 cbsd jexec jname='*' date produces a lot of outputs, the last line being:

+ task 'mode=new' 'logfile=/usr/jails/tmp/jexec_multiple.myjail.log.61670' 'client_id=myjail' 'autoflush=0' 'owner=jexec_multiple' /usr/bin/env 'NOCOLOR=1' /usr/local/bin/cbsd jexec 'jname=myjail' date

The mentioned log file does not exist.

@tm-up
Copy link
Author

tm-up commented Nov 2, 2019

I just observe that the cbsd process consumes approx. 25% CPU time (info from "top"- command).

The command cbsd jstart jname='*' or the equivalent jstop command works as expected.
UPDATE: I am not so sure whether this is completely true: The last three jails get started, but three processes are left over and their status 'Is': daemon: /usr/local/bin/cbsd[14806] (daemon).
The process 14806 is "<defunct>". (same for the other two daemon processes of cbsd).
When stopping those jails, I'll get the start-console output and then the jails stop.
There are more zombie processes belonging either to a "nice" command issued by CBSD or the above mentioned "daemon".

Any ideas what is going on here?

@olevole olevole self-assigned this Nov 3, 2019
@olevole
Copy link
Member

olevole commented Nov 3, 2019

can you post output for:

cbsd taskls

?

I also have an idea to rewrite 'cbsd task mode=new' and cbsdd with beanstalkd-based engine (minimalist broker, was demonstrated in https://www.bsdstore.ru/en/broker_driven_sample_ssi.html ).

However, the current implementation should handle simple operations like this.

@olevole
Copy link
Member

olevole commented Nov 3, 2019

Currently, all parallel/massive commands work according to the following principle:

  1. add cmd to queue,e.g:

cbsd task mode=new date

  1. cbsdd process (daemonized shell/cbsd, /usr/local/etc/rc.d/cbsdd) watch via kqueue(2) for taskd.sqlite modification ( ~cbsd/var/db/cbsdtaskd.sqlite ) and wakeup when any new task is coming.

  2. cbsdd execute commands with output to /tmp/*.log and update result/status.

  3. jexec/jstart/jstop/... command just 'poll' job status and read /tmp/*.log when status 'complete'

@olevole olevole closed this as completed in 18a6d9d Feb 4, 2020
@tm-up
Copy link
Author

tm-up commented Feb 22, 2020

The problem here was simple and stupid: cbsdd was not running...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants