Running a background q timer within an IPython session

[cmccarthy1]

Issue raised via email to fusion@kx.com, here for posterity

Questions

• [x ] Which operating system are you using (if Linux, please provide flavour of it, i.e RedHat, CentOS or Ubuntu), is it 32-bit, or 64-bit?

  • Behaviour should be os independent but have been replicated on the following OS

    $ uname -a
    Linux username 5.4.0-37-generic #41-Ubuntu SMP Wed Jun 3 18:57:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
    

• Which version of PyQ are you running? Please provide output of pyq --versions, if PyQ isn't operational, please provide Python interpreter version and PyQ version python -V; python3 -V; pip list | grep pyq:

  PyQ 5.0.0
  KDB+ 3.6 (2019.08.20) l64
  Python 3.8.2 (default, Apr 27 2020, 15:53:34)
  [GCC 9.3.0]
  

• Which version of kdb+ are you using, is it 32-bit or 64-bit?

  kdb+ 3.6 64-bit
  

• If on 64-bit, is your QLIC set? Please provide output env | grep QLIC on linux/macOS, or set|grep QLIC on Windows.

  • No QLIC is not set, this should not be relevant for the problem

• Did you use virtual environment to install PyQ? If not, why?

  • In the example PyQ was Installed in a conda environment

• Where is your QHOME? Please provide output env | grep QHOME on linux/macOS, or set|grep QHOME on Windows.

  $ env | grep QHOME
  QHOME=/home/conor/anaconda3/pyq_issue/q
  

• Do you use Conda? If so, what version?

  $ conda -V
  conda 4.8.3
  

Steps to reproduce the issue

The following minimal script was provided and should expose the behaviour, named test.q and executed as $ q test.q

t:([]id:();time:());
.ns.i:0;
.z.ts:{
    `t insert (.ns.i;.z.p);
    .ns.i+:1
 };
system"t 2000";
show t;
show string[.z.p]," : Breaking to python"
p)1
.pyq.run:.pyq.py `$1_string .Q.dd[hsym `$getenv `$"CONDA_PREFIX";`bin`pyq];
.pyq.run ("-m";"IPython")

Expected result

Expected behaviour from user was that the timer set initially before running the IPython session would continue to allow the table to be added too on the timer while in the IPython session.

Issue #10 indicates that once in an interactive python environment that the python REPL overrides the kdb main loop however which is likely the reason that the timers do not execute as expected?

Actual result

The timer is not executing in the background. Events can be triggered from direct calls to .z.ts through PyQ via

[1] q('.z.ts[]')

However this is not sufficient in the users use case

Workaround

If you know workaround, please provide it here.

A suitable workaround doesn't appear to be available hence the question. An async threaded python event loop can be run which allows for some level of flexibility but this use case requires that global variables be updated which seems to be outside the remit of the interface in this use case.

Any workarounds/insights into this would be greatly appreciated

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[github-actions]

Stale issue message