New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no event notifications with active redis-queue #6929

Open
mgoppold opened this Issue Sep 25, 2018 · 10 comments

Comments

3 participants
@mgoppold

mgoppold commented Sep 25, 2018

tine2.0 (2018.08.4) does not send event notifications when redis is active.

# redis-cli keys '*'
1) "TinebaseQueueData:3d2ccb8582efac66162c4449b7b6be877c6573be"
2) "TinebaseQueueQueue"
3) "TinebaseQueueData:05e4def4b6f32756e60636b1eaa14a430ee56b06"
4) "TinebaseQueueData:97777c024530599a6b2f34ca05438b2efe8a3b1b"
5) "TinebaseQueueData:8b58f32998e11766ef7ac1e026cc422eab74e4bd"
# grep 3d2ccb8582efac66162c4449b7b6be877c6573be tine20.log_TestNr12 
9140e 955e7 mgoppold - 2018-09-25T06:04:55+00:00 DEBUG (7): Tinebase_ActionQueue_Backend_Redis::send::219 queued job 3d2ccb8582efac66162c4449b7b6be877c6573be on queue TinebaseQueueQueue (datastructname: TinebaseQueueData)
# redis-cli dump 'TinebaseQueueQueue'
"\x0e\x01@\xb3\xb3\x00\x00\x00\x88\x00\x00\x00\x04\x00\x00(3d2ccb8582efac66162c4449b7b6be877c6573be*(05e4def4b6f32756e60636b1eaa14a430ee56b06*(97777c024530599a6b2f34ca05438b2efe8a3b1b*(8b58f32998e11766ef7ac1e026cc422eab74e4bd\xff\b\x00\xfb\xbf\x9e\x98A0\x04\xfb"
 # redis-cli dump 'TinebaseQueueData:3d2ccb8582efac66162c4449b7b6be877c6573be'
"\x04\x05\x05retry\xc0\x00\x04time\x132018-09-25 06:04:55\x04data\xc3[\xbe\x80\x00\x00h\xce\x1fa:3:{s:6:\"action\";s:31:\"Calendar\x01.s \x06\x0cEventNotifica ... 

tine20_6929.log

When redis is disabled event notifications will be sent.

@pschuele

This comment has been minimized.

Show comment
Hide comment
@pschuele

pschuele Sep 25, 2018

Member

you need to run the tine20-worker for the execution of the queue jobs, if you activate the queue config.
the easiest way to do this, is to install the tine20-worker deb package, which comes with systemd configuration.

we'll write a short howto on this in the near future.

Member

pschuele commented Sep 25, 2018

you need to run the tine20-worker for the execution of the queue jobs, if you activate the queue config.
the easiest way to do this, is to install the tine20-worker deb package, which comes with systemd configuration.

we'll write a short howto on this in the near future.

@pschuele pschuele self-assigned this Sep 25, 2018

@pschuele pschuele added this to To do in Tine 2.0 Documentation via automation Sep 25, 2018

@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 25, 2018

My Tine2.0 Service runs on openSUSE and so I could not use the dep out of the box, but:

wget https://packages.tine20.org/ubuntu/pool/stable/t/tine20/tine20-worker_2018.08.4-1.1804_all.deb
mkdir /tmp/tine20-worker
cd /tmp/tine20-worker
ar -xv ~/Downloads/tine20-worker_2018.08.4-1.1804_all.deb
tar xvJf data.tar.xz
scp -r ./etc/tine20/* root@tine20server:/srv/www/vhosts/etc/tine20
scp -r ./usr root@tine20server:/srv/www/vhosts/
scp ./lib/systemd/system/tine20-worker.service root@tine20server:/etc/systemd/system/

at tine20server:

chown -R wwwrun: /srv/www/vhosts/usr/
chown -R wwwrun: /srv/www/vhosts/etc/tine20/{actionQueue.ini,conf.d}
chmod -R a+r /srv/www/vhosts/etc/tine20/

Modifiy /etc/systemd/system/tine20-worker.service

WorkingDirectory=/srv/www/vhosts/tine20/tine20adm/
ExecStart=/usr/bin/php -d include_path=/srv/www/vhosts/tine20/ /srv/www/vhosts/usr/share/tine20/worker.php --config /srv/www/vhosts/etc/tine20/actionQueue.ini
User=wwwrun
Group=www

Modify /srv/www/vhosts/etc/tine20/actionQueue.ini

logfile = /srv/www/vhosts/tine20/tine20adm/worker.log

Start the service:

systemctl daemon-reload
systemctl enable tine20-worker.service
systemctl start tine20-worker.service

But the service wont start. The running cmdline is:

/usr/bin/php -d include_path=/srv/www/vhosts/tine20/ /srv/www/vhosts/usr/share/tine20/worker.php --config /srv/www/vhosts/etc/tine20/actionQueue.ini 

and the error:

PHP Fatal error:  Uncaught Error: Call to undefined function pcntl_signal() in /srv/www/vhosts/tine20/library/Console/Daemon.php:67
Stack trace:
#0 /srv/www/vhosts/tine20/Tinebase/ActionQueue/Worker.php(70): Console_Daemon->__construct(NULL)
#1 /srv/www/vhosts/usr/share/tine20/worker.php(19): Tinebase_ActionQueue_Worker->__construct()
#2 {main}
  thrown in /srv/www/vhosts/tine20/library/Console/Daemon.php on line 67

Fatal error: Uncaught Error: Call to undefined function pcntl_signal() in /srv/www/vhosts/tine20/library/Console/Daemon.php:67
Stack trace:
#0 /srv/www/vhosts/tine20/Tinebase/ActionQueue/Worker.php(70): Console_Daemon->__construct(NULL)
#1 /srv/www/vhosts/usr/share/tine20/worker.php(19): Tinebase_ActionQueue_Worker->__construct()
#2 {main}
  thrown in /srv/www/vhosts/tine20/library/Console/Daemon.php on line 67

mgoppold commented Sep 25, 2018

My Tine2.0 Service runs on openSUSE and so I could not use the dep out of the box, but:

wget https://packages.tine20.org/ubuntu/pool/stable/t/tine20/tine20-worker_2018.08.4-1.1804_all.deb
mkdir /tmp/tine20-worker
cd /tmp/tine20-worker
ar -xv ~/Downloads/tine20-worker_2018.08.4-1.1804_all.deb
tar xvJf data.tar.xz
scp -r ./etc/tine20/* root@tine20server:/srv/www/vhosts/etc/tine20
scp -r ./usr root@tine20server:/srv/www/vhosts/
scp ./lib/systemd/system/tine20-worker.service root@tine20server:/etc/systemd/system/

at tine20server:

chown -R wwwrun: /srv/www/vhosts/usr/
chown -R wwwrun: /srv/www/vhosts/etc/tine20/{actionQueue.ini,conf.d}
chmod -R a+r /srv/www/vhosts/etc/tine20/

Modifiy /etc/systemd/system/tine20-worker.service

WorkingDirectory=/srv/www/vhosts/tine20/tine20adm/
ExecStart=/usr/bin/php -d include_path=/srv/www/vhosts/tine20/ /srv/www/vhosts/usr/share/tine20/worker.php --config /srv/www/vhosts/etc/tine20/actionQueue.ini
User=wwwrun
Group=www

Modify /srv/www/vhosts/etc/tine20/actionQueue.ini

logfile = /srv/www/vhosts/tine20/tine20adm/worker.log

Start the service:

systemctl daemon-reload
systemctl enable tine20-worker.service
systemctl start tine20-worker.service

But the service wont start. The running cmdline is:

/usr/bin/php -d include_path=/srv/www/vhosts/tine20/ /srv/www/vhosts/usr/share/tine20/worker.php --config /srv/www/vhosts/etc/tine20/actionQueue.ini 

and the error:

PHP Fatal error:  Uncaught Error: Call to undefined function pcntl_signal() in /srv/www/vhosts/tine20/library/Console/Daemon.php:67
Stack trace:
#0 /srv/www/vhosts/tine20/Tinebase/ActionQueue/Worker.php(70): Console_Daemon->__construct(NULL)
#1 /srv/www/vhosts/usr/share/tine20/worker.php(19): Tinebase_ActionQueue_Worker->__construct()
#2 {main}
  thrown in /srv/www/vhosts/tine20/library/Console/Daemon.php on line 67

Fatal error: Uncaught Error: Call to undefined function pcntl_signal() in /srv/www/vhosts/tine20/library/Console/Daemon.php:67
Stack trace:
#0 /srv/www/vhosts/tine20/Tinebase/ActionQueue/Worker.php(70): Console_Daemon->__construct(NULL)
#1 /srv/www/vhosts/usr/share/tine20/worker.php(19): Tinebase_ActionQueue_Worker->__construct()
#2 {main}
  thrown in /srv/www/vhosts/tine20/library/Console/Daemon.php on line 67
@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 25, 2018

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

*/5 * * * * root /usr/bin/php -d include_path=.:/srv/www/vhosts/tine20/:/srv/www/vhosts/tine20/library/:/etc/tine20 /srv/www/vhosts/tine20/tine20.php --method Tinebase.triggerAsyncEvents >> /srv/www/vhosts/tine20/tine20adm/tine20_async_cron.log; touch /srv/www/vhosts/tine20/tine20adm/tine20_async_cron.log

mgoppold commented Sep 25, 2018

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

*/5 * * * * root /usr/bin/php -d include_path=.:/srv/www/vhosts/tine20/:/srv/www/vhosts/tine20/library/:/etc/tine20 /srv/www/vhosts/tine20/tine20.php --method Tinebase.triggerAsyncEvents >> /srv/www/vhosts/tine20/tine20adm/tine20_async_cron.log; touch /srv/www/vhosts/tine20/tine20adm/tine20_async_cron.log
@pschuele

This comment has been minimized.

Show comment
Hide comment
@pschuele

pschuele Sep 25, 2018

Member

nice, thanks for the howto!

PHP Fatal error: Uncaught Error: Call to undefined function pcntl_signal()

maybe you are missing the "pcntl" php module. we should add this as a requirement for the worker service.

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

no, this is still needed! the worker just checks the redis queue and runs the jobs that are out into it.

Member

pschuele commented Sep 25, 2018

nice, thanks for the howto!

PHP Fatal error: Uncaught Error: Call to undefined function pcntl_signal()

maybe you are missing the "pcntl" php module. we should add this as a requirement for the worker service.

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

no, this is still needed! the worker just checks the redis queue and runs the jobs that are out into it.

@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 25, 2018

Ok I have got it working :-)

After 'zypper install php7-pcntl` and a few adjustments (group wwwrun->www, chown Config-Files) the service is running and notifications where send.

Thank you for pointing me in the right direction.

mgoppold commented Sep 25, 2018

Ok I have got it working :-)

After 'zypper install php7-pcntl` and a few adjustments (group wwwrun->www, chown Config-Files) the service is running and notifications where send.

Thank you for pointing me in the right direction.

@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 25, 2018

nice, thanks for the howto!

PHP Fatal error: Uncaught Error: Call to undefined function pcntl_signal()

maybe you are missing the "pcntl" php module. we should add this as a requirement for the worker service.

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

no, this is still needed! the worker just checks the redis queue and runs the jobs that are out into it.

In order not to have cron and systemd services here is a systemd.timer for the async jobs:

tine20-async.service.txt
tine20-async.timer.txt

scp tine20-async.*.txt root@tine20server:/etc/systemd/system/

mv /etc/systemd/system/tine20-async.service.txt /etc/systemd/system/tine20-async.service
mv /etc/systemd/system/tine20-async.timer.txt /etc/systemd/system/tine20-async.timer
systemctl daemon-reload
systemctl enable tine20-async.timer
systemctl start tine20-async.timer
systemctl list-timers 
NEXT                         LEFT         LAST                         PASSED       UNIT                         ACTIVATES
Di 2018-09-25 13:10:00 CEST  3min 18s ago Di 2018-09-25 13:10:01 CEST  3min 17s ago tine20-async.timer           tine20-async.service

mgoppold commented Sep 25, 2018

nice, thanks for the howto!

PHP Fatal error: Uncaught Error: Call to undefined function pcntl_signal()

maybe you are missing the "pcntl" php module. we should add this as a requirement for the worker service.

Does the service completely replace the earlier "Asynchronous jobs" from cron.d?

no, this is still needed! the worker just checks the redis queue and runs the jobs that are out into it.

In order not to have cron and systemd services here is a systemd.timer for the async jobs:

tine20-async.service.txt
tine20-async.timer.txt

scp tine20-async.*.txt root@tine20server:/etc/systemd/system/

mv /etc/systemd/system/tine20-async.service.txt /etc/systemd/system/tine20-async.service
mv /etc/systemd/system/tine20-async.timer.txt /etc/systemd/system/tine20-async.timer
systemctl daemon-reload
systemctl enable tine20-async.timer
systemctl start tine20-async.timer
systemctl list-timers 
NEXT                         LEFT         LAST                         PASSED       UNIT                         ACTIVATES
Di 2018-09-25 13:10:00 CEST  3min 18s ago Di 2018-09-25 13:10:01 CEST  3min 17s ago tine20-async.timer           tine20-async.service

@pschuele

This comment has been minimized.

Show comment
Hide comment
@pschuele

pschuele Sep 26, 2018

Member

Ok I have got it working :-)

great :)

In order not to have cron and systemd services here is a systemd.timer for the async jobs:

thanks for sharing that. I'll link to that from the wiki.

Member

pschuele commented Sep 26, 2018

Ok I have got it working :-)

great :)

In order not to have cron and systemd services here is a systemd.timer for the async jobs:

thanks for sharing that. I'll link to that from the wiki.

@lab-at-nohl

This comment has been minimized.

Show comment
Hide comment
@lab-at-nohl

lab-at-nohl Sep 29, 2018

Contributor

I wonder, if we can have the systemd’s service and timer in a single rpm? Or at least a script for automatization.

Probably for a standards compliant rpm their is no need for distinction between openSUSE and RH-based flavors. Of course a general rpm should use global paths; to make worker.php known (which has a random location) you may use a symbolic link in /etc/tine20.

Contributor

lab-at-nohl commented Sep 29, 2018

I wonder, if we can have the systemd’s service and timer in a single rpm? Or at least a script for automatization.

Probably for a standards compliant rpm their is no need for distinction between openSUSE and RH-based flavors. Of course a general rpm should use global paths; to make worker.php known (which has a random location) you may use a symbolic link in /etc/tine20.

@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 30, 2018

Sure, there is no reason not to put both in one rpm:

%install
...
install -c -m 644 tine20-async.timer  ${RPM_BUILD_ROOT}/%{_unitdir}
install -c -m 644 tine20-async.service  ${RPM_BUILD_ROOT}/%{_unitdir}
...
%post
test -x /bin/systemctl && /bin/systemctl daemon-reload || :
test -x /bin/systemctl && /bin/systemctl enable  tine20-async.timer &&  /bin/systemctl start tine20-async.timer

%postun
test -x /bin/systemctl && /bin/systemctl stop tine20-async.timer && /bin/systemctl disbale tine20-async.timer
test -x /bin/systemctl && /bin/systemctl daemon-reload || :


%files
...
%{_unitdir}/tine20-async.*

this install the timer and activates them.

mgoppold commented Sep 30, 2018

Sure, there is no reason not to put both in one rpm:

%install
...
install -c -m 644 tine20-async.timer  ${RPM_BUILD_ROOT}/%{_unitdir}
install -c -m 644 tine20-async.service  ${RPM_BUILD_ROOT}/%{_unitdir}
...
%post
test -x /bin/systemctl && /bin/systemctl daemon-reload || :
test -x /bin/systemctl && /bin/systemctl enable  tine20-async.timer &&  /bin/systemctl start tine20-async.timer

%postun
test -x /bin/systemctl && /bin/systemctl stop tine20-async.timer && /bin/systemctl disbale tine20-async.timer
test -x /bin/systemctl && /bin/systemctl daemon-reload || :


%files
...
%{_unitdir}/tine20-async.*

this install the timer and activates them.

@mgoppold

This comment has been minimized.

Show comment
Hide comment
@mgoppold

mgoppold Sep 30, 2018

Probably for a standards compliant rpm their is no need for distinction between openSUSE and RH-based flavors. Of course a general rpm should use global paths; to make worker.php known (which has a random location) you may use a symbolic link in /etc/tine20.

sure, what I want to show is to activate the service for users they don't install tine20 with rpm.

by now, you only provide dep's or not?

mgoppold commented Sep 30, 2018

Probably for a standards compliant rpm their is no need for distinction between openSUSE and RH-based flavors. Of course a general rpm should use global paths; to make worker.php known (which has a random location) you may use a symbolic link in /etc/tine20.

sure, what I want to show is to activate the service for users they don't install tine20 with rpm.

by now, you only provide dep's or not?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment