Create version of PyQ for Windows? #1

rogermcintosh · 2015-04-29T01:31:48Z

Hi
I am evaluating kdb+/q for use in managing large data volumes for an investment process and was keen on utilising PyQ to help manage data wrangling into a kdb database. I am restricted to operating on Windows and noticed that the pip install or build from source won't work as it is written only for Unix variants OSs. Is there any chance this could be extended to Windows? I'd have a crack at it myself, but my programming skills not that extensive .. Would it require much work to adjust to compile on WIndows? I am running Python 3.4

abalkin · 2015-07-02T17:16:40Z

We have no plans to support Windows at the moment. The major supported platforms are MacOS and Linux while we had some success with Solaris and FreeBSD.

The code itself should compile without any problems on Windows, but some changes will be required
to setup.py.

sashkab · 2017-08-30T19:49:15Z

Experimental support for Windows has landed with the PyQ 4.1.0.

Ensure that kdb+ is installed under C:\q, or QHOME environment variable properly set.

Then, you can install PyQ on Windows using

pip install -U pyq

You can start PyQ by running

c:\q\w32\q.exe python.q

Note, you will have to press ^Z and then Enter key in order to get into Python REPL. This is known limitation of this version.

You can run tests too: first install required packages:

pip install pytest pytest-pyq

Then run:

set QBIN=c:\q\w32\q.exe
c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul

We appreciate all bug reports and any feedback.

bevinahal · 2017-09-04T09:24:22Z

Thanks so much for this release - this is a very good feature.

My installation fails with below error. Is this because I'm using 64-bit q.exe version?
I did run vcvarsall.bat x64 before calling "python setup.py install"

Python version : 3.5.2
KDB version : 3.1

Error -

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIB
ATH:C:\Users\44025173\AppData\Local\Continuum\Anaconda3\libs /LIBPATH:C:\Users\44025173\AppData\Local\Continuum\Anaconda3\PCbuild\amd64 "/LIBPATH:C:\Progr
m Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\P
ogram Files (x86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program
Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x
6)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x64" "/LIBPATH:C:\Program File
 (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x64" /LIBPATH:C:\SYBASE\OCS-12_5\lib /EXPO
T:PyInit__k build\temp.win-amd64-3.5\Release\src/pyq/_k.obj /OUT:build\lib.win-amd64-3.5\pyq\_k.cp35-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.5\Releas
\src/pyq\_k.cp35-win_amd64.lib src\pyq\kx\w64\q.lib
_k.obj : warning LNK4197: export 'PyInit__k' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.5\Release\src/pyq\_k.cp35-win_amd64.lib and object build\temp.win-amd64-3.5\Release\src/pyq\_k.cp35-win_amd64.e
p
_k.obj : error LNK2001: unresolved external symbol clr
build\lib.win-amd64-3.5\pyq\_k.cp35-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1120

abalkin · 2017-09-04T12:15:41Z

Hi @bevinahal,

Thank you for your report. I don't think 64bit is an issue here. More likely it is your use of kdb+ 3.1. We only fully tested PyQ with the latest (3.5, 32bit) kdb+, but it does compile with (3.5, 64bit) as well.

The problem is that with kdb+ < 3.5, PyQ uses an undocumented clr() function which apparently is not included in q.lib. I see the following options for you

Upgrade to kdb+ 3.5.
Add void clr(void) definition to q.lib. Your Kx support may be able to help you with that.
Comment out line 158 in _k.c.

If you decide to take option (3), see "Installing from Source Code" instructions.
Please let us know if any of these solutions work for you.

abalkin · 2017-09-14T17:00:06Z

Dear @chenkangcrack,

Thank you for your report. For the 4.1 release, we've only tested PyQ on Windows with Python 3.6. However, it looks like only a minor change is needed to make it compile under Python 2.7.

I am attaching the necessary patch. To apply, clone this repository, download the attachment and run

git apply 0001-BUG-946-Fixed-compilation-errors-for-Python-2.7-on-W.patch.txt

You should then be able to install pyq with

pip install .

Please let us know if you have any other issues. We will include these changes in the next bug-fix release.

0001-BUG-946-Fixed-compilation-errors-for-Python-2.7-on-W.patch.txt

sashkab · 2017-09-15T12:17:44Z

@chenkangcrack Please try to start PyQ as described above by runningc:\q\w32\q.exe python.q (replace c:\q\w32 with path to your kdb+ installation).

When installing PyQ, please use Command Prompt, not git/mingw shell. You might need to install Visual Studio 9 for Python if using Python 2.7.x, and pip will instruct you what to do.

abalkin · 2017-09-15T12:29:57Z

Hi @chenkangcrack,

Thank you for trying the patch. Please note that PyQ runs Python inside a kdb+ process and cannot be loaded in a standard Python. To start an interactive session, please enter the following command:

c:\q\w32\q.exe python.q

As explained above, you will have to press Control+Z and then Enter keys to break out of the kdb+ REPL and start entering Python commands. (When running python scripts, redirect STDIN to nul.)

To make sure you've succeeded in building PyQ, please run the included test suit:

pip install pytest pytest-pyq
set QBIN=c:\q\w32\q.exe
c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul

ghlian · 2017-09-16T14:03:56Z

This is error I got.
win 10 64bit
python 36 32bit
compile the pyq successfully but cannot launch it.

(python36_32) C:\Users\ligu1>c:\q\w32\q.exe python.q
KDB+ 3.5 2017.09.06 Copyright (C) 1993-2017 Kx Systems
w32/ 4()core 4095MB ghlian liandelllaptop 192.168.43.33 NONEXPIRE

Welcome to kdb+ 32bit edition
For support please see http://groups.google.com/d/forum/personal-kdbplus
Tutorials can be found at http://code.kx.com/wiki/Tutorials
To exit, type \\
To remove this startup msg, edit q.q
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000060c0 (most recent call first):

ghlian · 2017-09-16T15:19:02Z

After failing to launch it in python36 above (the pyq crashed)

I tried to do it in python 27 as you described, got the error and cannot start it.

(python27_32) C:\Users\ligu1\pyq>pip install .
Processing c:\users\ligu1\pyq
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Exception:
Traceback (most recent call last):
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "F:\Anaconda_3_64\envs\python27_32\lib\site-packages\pip\req\req_install.py", line 919, in install
    prepend_root(filename), egg_info_dir)
  File "F:\Anaconda_3_64\envs\python27_32\lib\ntpath.py", line 529, in relpath
    % (path_prefix, start_prefix))
ValueError: path is on drive C:, start on drive F:

Is it because my python is installed in F driver?

Thanks a lot.

abalkin · 2017-09-16T18:03:42Z

Dear @ghlian,

It looks like you are using conda. We have not tested PyQ with it. I suggest that you install the official CPython distribution from https://www.python.org/downloads/windows/. For the 32-bit version, choose "Download Windows x86 MSI installer".

abalkin · 2017-09-18T15:44:33Z

Dear @ghlian,

I've just downloaded Miniconda3 (32bit) and successfully installed pyq:

(C:\Users\a\Miniconda3) C:\Users\a>pip install --no-binary pyq pyq
Collecting pyq
  Downloading pyq-4.1.0.post1.tar.gz (93kB)
    100% |████████████████████████████████| 102kB 655kB/s
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Successfully installed pyq-4.1.0

(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q python.q
KDB+ 3.5 2017.03.28 Copyright (C) 1993-2017 Kx Systems
w32/ 1()core 4095MB a enlnt-win 10.200.144.21 NONEXPIRE

Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 14:16:49) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
^Z

>>> 1+2
3
>>> from pyq import q
>>> q.til(5)
k('0 1 2 3 4')
>>> exit()

There was a slight problem running tests: the standard pytest package had a trouble parsing Anaconda Python version:

(C:\Users\a\Miniconda3) C:\Users\a>pip install pytest pytest-pyq
..
(C:\Users\a\Miniconda3) C:\Users\a>set QBIN=c:\q\w32\q.exe
(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q.exe python.q -mpytest --pyargs pyq < nul
..
INTERNALERROR> ValueError: failed to parse CPython sys.version: '3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 14:16:49) [MSC v.1900 32 bit (Intel)]'

As a work-around, I ran pytest in a "quiet" mode:

(C:\Users\a\Miniconda3) C:\Users\a>c:\q\w32\q.exe python.q -mpytest --quiet --pyargs pyq < nul
KDB+ 3.5 2017.03.28 Copyright (C) 1993-2017 Kx Systems
w32/ 1()core 4095MB a enlnt-win 10.200.144.21 NONEXPIRE

s................................................................................................................................................................................................................................s...........................................s........................................................................................................................................s....................................................s................................................................x..................................................................................................s....ssssssssssssssssssssssssssss.......s.......................................................................................................s.........................................................................................................................................................................................ss.....
921 passed, 38 skipped, 1 xfailed in 3.59 seconds

Overall, it seems to work for me. I suspect you have some custom modifications in your Anaconda installation. Can you try to install a fresh version of Miniconda3? Please make sure that you use 32bit Python with 32bit q.

ghlian · 2017-09-18T22:52:11Z

Thanks very much. I did get it installed and run when python is installed in C driver.

But when it is installed in other driver. You get the error.

(python36_32) C:\Users\ligu1\Desktop>pip install --no-binary pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... done
Exception:
Traceback (most recent call last):
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\site-packages\pip\req\req_install.py", line 919, in install
    prepend_root(filename), egg_info_dir)
  File "F:\Python\Anaconda_36_64\envs\python36_32\lib\ntpath.py", line 585, in relpath
    path_drive, start_drive))
ValueError: path is on mount 'C:', start on mount 'F:'

abalkin · 2017-09-18T23:50:13Z

@ghlian, the problem that you see does not look like anything specific to pyq. It looks like an issue with pip. I don't have a Windows setup with multiple drives at the moment, but I would suggest that you try to set the TMPDIR environment variable to some path on the F: drive, e.g. F:\tmp, assuming that such directory exists and you have permissions to read and write into it.

ghlian · 2017-09-18T23:58:27Z

Thank you!!
I will leave it in C driver for now.
How to launch it pyq in Jupyter in Windows? I guess the way for linux (pyq -m notebook) does not fit Windows because pyq is not defined.

abalkin · 2017-09-19T00:04:53Z

Try

c:\q\w32\q python.q -mnotebook

Make sure that there is no space between -m and the name of the python module. Alternatively, you can add @ after -m:

c:\q\w32\q python.q -m@ notebook

This is necessary because q tries to interpret any single-letter option as its own.

You may also want to install the pyq.kernel add-on. See https://pypi.python.org/pypi/pyq-kernel.

ghlian · 2017-09-19T00:54:49Z

Is there any problem with the specification here

"kernelspecs": {
    "pyq_3": {
      "resource_dir": "C:\\Users\\ligu1\\AppData\\Roaming\\jupyter\\kernels\\pyq_3",
      "spec": {
        "argv": [
          null,
          "pyq-kernel.p",
          "1024",
          "65535",
          "{connection_file}"
        ],
        "env": {
          "QHOME": "C:\\q"
        },
        "display_name": "PyQ 3",
        "language": "python"
      }
    },

The jupyter was launched but the kernel of pyq cannot be started with the error

Traceback (most recent call last):
  File "C:\python_36_64_raw\lib\site-packages\notebook\base\handlers.py", line 540, in wrapper
    result = yield gen.maybe_future(method(self, *args, **kwargs))
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\handlers.py", line 75, in post
    type=mtype))
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 79, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1063, in run
    yielded = self.gen.throw(*exc_info)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 92, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 1055, in run
    value = future.result()
  File "C:\python_36_64_raw\lib\site-packages\tornado\concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "C:\python_36_64_raw\lib\site-packages\tornado\gen.py", line 307, in wrapper
    yielded = next(result)
  File "C:\python_36_64_raw\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 123, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 244, in start_kernel
    kernel_cmd = self.format_kernel_cmd(extra_arguments=extra_arguments)
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 196, in format_kernel_cmd
    return [ pat.sub(from_ns, arg) for arg in cmd ]
  File "C:\python_36_64_raw\lib\site-packages\jupyter_client\manager.py", line 196, in <listcomp>
    return [ pat.sub(from_ns, arg) for arg in cmd ]
TypeError: expected string or bytes-like object

hzadonis · 2017-09-19T03:12:06Z

Exception report at Windows 10, KDB+ 3.5 2017.03.15

D:\Program\Python36\Scripts>where q
C:\q\w32\q.exe

D:\Program\Python36\Scripts>D:\Program\Python36\Scripts\pip.exe install --no-binary pyq pyq
Collecting pyq
  Using cached pyq-4.1.0.post1.tar.gz
Skipping bdist_wheel for pyq, due to binaries being disabled for it.
Installing collected packages: pyq
  Running setup.py install for pyq ... error
Exception:
Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 73, in console_to_str
    return s.decode(sys.__stdout__.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\program\python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\program\python36\lib\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "d:\program\python36\lib\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "d:\program\python36\lib\site-packages\pip\req\req_install.py", line 878, in install
    spinner=spinner,
  File "d:\program\python36\lib\site-packages\pip\utils\__init__.py", line 676, in call_subprocess
    line = console_to_str(proc.stdout.readline())
  File "d:\program\python36\lib\site-packages\pip\compat\__init__.py", line 75, in console_to_str
    return s.decode('utf_8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 26: invalid start byte

abalkin · 2017-09-19T14:27:09Z

@hzadonis, it looks like your pip process is having trouble displaying the progress bar. Please try running it in silent mode by adding the -qqq option on the command line. You can also try setting the PYTHONIOENCODING environment variable to utf-8.

abalkin · 2017-09-19T17:47:40Z

@ghlian -

Is there any problem with the specification here?

"kernelspecs": {
  "pyq_3": {
    "resource_dir": "C:\Users\ligu1\AppData\Roaming\jupyter\kernels\pyq_3",
    "spec": {
      "argv": [
        null,
        "pyq-kernel.p",
        "1024",
        "65535",
        "{connection_file}"
    ],
    "env": {
      "QHOME": "C:\q"
    },
    "display_name": "PyQ 3",
    "language": "python"
  }
},

Yes, first I believe json requires that backslashes are escaped. (Double every backslash in the file paths.) Second, make sure the environment variable QBIN is set to the full path to q.exe before you run q python.q -mpyq.kernel install. Alternatively, edit the kernel.json file and replace null in argv list with the path to q.exe. Finally, I am not sure it causes any harm, but the trailing comma is unnecessary.

abalkin · 2017-09-19T19:18:39Z

Installing Jupyter kernel

Since we have not ported the pyq executable to the Windows platform yet, setting up a working PyQ environment on Windows requires several manual steps.

First, it is strongly recommended to use a dedicated Python virtual environment and install q in %VIRTUAL_ENV%. Assuming that you have downloaded windows.zip from https://kx.com/download/ in your Downloads folder, enter the following commands:

python -mvenv py36
py36\Scripts\activate.bat
set QHOME=%VIRTUAL_ENV%\q
"C:\Program Files\7-Zip\7z.exe" x -y -o%VIRTUAL_ENV% %HOMEPATH%\Downloads\windows.zip
del %QHOME%\q.q
set PYTHONPATH=%VIRTUAL_ENV%\lib\site-packages
set QBIN=%QHOME%\w32\q.exe

Now you should be able to install jupyter, pyq and pyq-kernel in one commad

pip install jupyter pyq pyq-kernel

Finally, to install pyq kernel specs, run

%QBIN% python.q -mpyq.kernel install

If everything is successful, you should see pyq_3 listed in the kernelspec list:

>jupyter kernelspec list
Available kernels:
  pyq_3      C:\Users\a\AppData\Roaming\jupyter\kernels\pyq_3
  python3    c:\users\a\py36\share\jupyter\kernels\python3

Now, start the notebook server

jupyter-notebook

and select "PyQ 3" from the "New" menu. For examples of what can be done in a PyQ notebook, please see my presentation at https://youtu.be/v2UoP0l6mOw.

ghlian · 2017-09-19T21:47:26Z

Thank. I figured out the problem is the file "pyq-kernel.p" is not accessible by pyq. Specifying the full path solves the problem.

ghlian · 2017-09-19T21:49:42Z

@abalkin
When pyq is launch and q is imported, is there any way that we can use other IDE sublime or q-studio to connect to the same q session in this pyq? It will be helpful and efficient to have this figure for working.

abalkin · 2017-09-20T01:13:12Z

@ghlian,

Yes, as long as you don't load python.q, which makes your q process enter into Python REPL, you should be able to have normal kdb+ connections and run python code. The easiest way to achieve that is to install Jupyter and pyq-kernel. I posted Windows instructions in my earlier comment.

Once you have pyq-kernel running inside your q process, you can connect a Jupyter notebook or an IPython console to it as well as qcon or any q IDE that can talk to a kdb+ server.

Create lgtm.yml (#1) * Create lgtm.yml Basic lgtm configuration based on the example [1]. [1]: https://lgtm.com/help/lgtm/lgtm.yml-configuration-file#example-lgtm.yml-file * Removed the configure command in lgtm.yml * Replace lgtm.yml with a simple version Thanks, Arthur Baars. * Attempt a full build in lgtm.yml Update pyq.c Generate C coverage Pass compiler defaults to build exe compiler Pass coverage flags via CC Restore LDFLAGS. Separate travis build commands in a separate file. Added enlnt channel to the conda install command. Fixed flags. More requirements Fix requirements Fix build More C coverage Add 64-bit environment Send license Fix conda 32-bit settings. Try using kdb+ from conda Added osx builds Update build.sh Added diagnostic. BUG Resolves #1008 Added logic to find Python DLL from sysconfig. Clean-up * Use new style of conda initialization * Do not download kdb outside of getkdb.sh script Allow failures on osx Do not download PSF python Specify language c Fix TST #703 Fixed tests that required VIRTUAL_ENV to be set. Fix conda download on macOS fix Fix base64 decode option Do not use -lgcov on macOS Fix pyq.c

jtzharry · 2018-08-09T04:49:51Z

Wonder if anyone see this error? Windows 10 64bit KDB 32bit newest version. Could it be that the visual studio C++ build tools has some problem? Thanks!

_k.obj : warning LNK4197: export 'PyInit__k' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.lib and object build\temp.win-amd64-3.6\Release\src/pyq\_k.cp36-win_amd64.exp
_k.obj : error LNK2001: unresolved external symbol ka
_k.obj : error LNK2001: unresolved external symbol kh
_k.obj : error LNK2001: unresolved external symbol ki
_k.obj : error LNK2001: unresolved external symbol kz
_k.obj : error LNK2001: unresolved external symbol kg
_k.obj : error LNK2001: unresolved external symbol js
_k.obj : error LNK2001: unresolved external symbol sn
_k.obj : error LNK2001: unresolved external symbol kc
_k.obj : error LNK2001: unresolved external symbol ku
_k.obj : error LNK2001: unresolved external symbol krr
_k.obj : error LNK2001: unresolved external symbol sd0
_k.obj : error LNK2001: unresolved external symbol dl
_k.obj : error LNK2001: unresolved external symbol jv
_k.obj : error LNK2001: unresolved external symbol ymd
_k.obj : error LNK2001: unresolved external symbol ee
_k.obj : error LNK2001: unresolved external symbol ktj
_k.obj : error LNK2001: unresolved external symbol kb
_k.obj : error LNK2001: unresolved external symbol jk
_k.obj : error LNK2001: unresolved external symbol ke
_k.obj : error LNK2001: unresolved external symbol d9
_k.obj : error LNK2001: unresolved external symbol kj
_k.obj : error LNK2001: unresolved external symbol m9
_k.obj : error LNK2001: unresolved external symbol sd1
_k.obj : error LNK2001: unresolved external symbol dot
_k.obj : error LNK2001: unresolved external symbol r0
_k.obj : error LNK2001: unresolved external symbol ktd
_k.obj : error LNK2001: unresolved external symbol k
_k.obj : error LNK2001: unresolved external symbol r1
_k.obj : error LNK2001: unresolved external symbol setm
_k.obj : error LNK2001: unresolved external symbol ss
_k.obj : error LNK2001: unresolved external symbol b9
_k.obj : error LNK2001: unresolved external symbol dj
_k.obj : error LNK2001: unresolved external symbol xT
_k.obj : error LNK2001: unresolved external symbol ktn
_k.obj : error LNK2001: unresolved external symbol ks
_k.obj : error LNK2001: unresolved external symbol kd
_k.obj : error LNK2001: unresolved external symbol xD
_k.obj : error LNK2001: unresolved external symbol kf
_k.obj : error LNK2001: unresolved external symbol ja
_k.obj : error LNK2001: unresolved external symbol kt
_k.obj : error LNK2001: unresolved external symbol kpn
_k.obj : error LNK2001: unresolved external symbol okx
_k.obj : error LNK2001: unresolved external symbol knk
build\lib.win-amd64-3.6\pyq\_k.cp36-win_amd64.pyd : fatal error LNK1120: 43 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1120

sashkab · 2018-08-09T12:19:00Z

@jtzharry Could you please share the command you used to get this error message? Did you use virtual environment? Which version of Python did you use? Which version of kdb+ did you use (i.e. 32-bit or 64-bit)?

jtzharry · 2018-08-10T01:51:47Z

@sashkab
Thanks, I am using windows 10 64 bit and kdb 32bit.

I am using this script:
python -mvenv py36
py36\Scripts\activate.bat
set QHOME=%VIRTUAL_ENV%\q
"C:\Program Files\7-Zip\7z.exe" x -y -o%VIRTUAL_ENV% %HOMEPATH%\Downloads\windows.zip
del %QHOME%\q.q
set QBIN=%QHOME%\w32\q.exe

pip install --pre pyq

I had an issue earlier saying visual c++ build tools is not installed, then I download from here:
https://download.microsoft.com/download/5/f/7/5f7acaeb-8363-451f-9425-68a90f98b238/visualcppbuildtools_full.exe

sashkab · 2018-08-10T12:11:04Z

@jtzharry You don't need to use pip install —pre pyq any longer. Please use pip install pyq instead. Is your Python is also 32-bit version?

jtzharry · 2018-08-11T01:10:35Z

@sashkab Thanks, changed to pip install pyq does not help, I think I am using 64bit Python.

sashkab · 2018-08-11T01:26:27Z

@jtzharry

I think I am using 64bit Python.

You need to use 32-bit python if you use 32-bit kdb+.

You can download it from python.org (select Windows x86 executable installer).

jtzharry · 2018-08-22T02:03:55Z

@sashkab Thanks and sorry for the late reply, I am able to install pyq through a 32-bit virtualenv with the 32BIT turned on.

sashkab · 2018-08-22T12:04:02Z

@jtzharry thanks for confirmation.

wjm3333 · 2018-10-10T12:28:24Z

I'm using Windows 10 (64-bit) Chinese version, Python 3.6 (32-bit) and the free version of q which is 32-bit. I've successfully installed pyq ,But when I run %QBIN% python.q，How can it not work normally?
Every other line has no result.

sashkab · 2018-10-10T15:21:05Z

@wjm3333 Could you please type after starting PyQ following:

>>> import pyq
>>> pyq.versions()

wjm3333 · 2018-10-10T16:32:04Z

Every other line has no result,Can't appear.

wjm3333 · 2018-10-10T16:35:06Z

I am using this script:

python -mvenv py36
py36\Scripts\activate.bat
set QHOME=%VIRTUAL_ENV%\q
"C:\Program Files\7-Zip\7z.exe" x -y -o%VIRTUAL_ENV% %HOMEPATH%\Downloads\windows.zip
del %QHOME%\q.q
set QBIN=%QHOME%\w32\q.exe

pip install pyq

wjm3333 · 2018-10-10T16:44:55Z

C:\Users\Administrator>python
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 02:47:15) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
`>>>import pyq`
Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyq\__init__.py", line 22, in <module>
    from ._k import K as _K, error as kerr, Q_VERSION, Q_DATE, Q_OS
ImportError: DLL load failed: 找不到指定的模块。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyq\__init__.py", line 27, in <module>
    if not hasattr(ctypes.CDLL(None), 'b9'):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
TypeError: LoadLibrary() argument 1 must be str, not None
`>>>from pyq import q`
Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyq\__init__.py", line 22, in <module>
    from ._k import K as _K, error as kerr, Q_VERSION, Q_DATE, Q_OS
ImportError: DLL load failed: 找不到指定的模块。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyq\__init__.py", line 27, in <module>
    if not hasattr(ctypes.CDLL(None), 'b9'):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
TypeError: LoadLibrary() argument 1 must be str, not None
>>>

sashkab · 2018-10-10T17:37:29Z

Your error is expected. You can launch PyQ on windows only by following instructions outlined above. Importing pyq in vanilla Python is not supported.

wjm3333 · 2018-10-11T02:27:57Z

Thank you for your reply. I did as you suggested. The result of starting PYQ did not report an error, but the input of 9 + 7 operation could not display the result.

>>> 9+7

>>> import pyq

>>> pyq.versions()

>>> 9+3

>>>

All inputs cannot display results on the next line，why?

jmsblah · 2018-12-05T16:11:37Z

Your error is expected. You can launch PyQ on windows only by following instructions outlined above. Importing pyq in vanilla Python is not supported.

I am trying to run pyq in windows as instructions, but I am getting the following error:

c:\Users\user\kdb\w64>q python.q
KDB+ 3.4 2016.09.26 Copyright (C) 1993-2016 Kx Systems
w64/ 24()core 65457MB xxxxxx xxxxxxx xxxxx NONEXPIRE  xxxxxxxxxx

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\Anaconda3\lib\site-packages\pyq\__init__.py", line 22, in <module>
    from ._k import K as _K, error as kerr, Q_VERSION, Q_DATE, Q_OS
ImportError: DLL load failed: The specified procedure could not be found.
k){0N!x y}
'python
@
"k"
"p)from pyq import q"
q))

Can someone help?

sashkab · 2018-12-06T13:35:05Z

@jmsblah please uninstall pyq pip uninstall pyq. Then please run pip install pyq and provide output.

decranfield1 · 2019-12-01T17:50:37Z

(base) C:\Users\deep_bedford>C:\q\w64\q.exe python.q
KDB+ 3.6 2019.09.19 Copyright (C) 1993-2019 Kx Systems
w64/ 8(16)core 16235MB deep_bedford desktop-vpmduit 192.168.0.14 EXPIRE 2020.11.28 KOD #4168058

Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000cce0 (most recent call first):

please help me to resolve the issue

sashkab · 2019-12-02T01:55:50Z

@decranfield1 It seems there are issues with your Python installation. Perhaps, you should re-create virtual environment and try re-installing PyQ again, following our instructions above.

decranfield1 · 2019-12-02T09:29:37Z

(base) C:\Users\deep_bedford>C:\Users\deep_bedford\Anaconda3\q\w64\q.exe python.q ^Z
KDB+ 3.6 2019.08.20 Copyright (C) 1993-2019 Kx Systems
w64/ 8(16)core 16235MB deep_bedford desktop-vpmduit 192.168.0.14 EXPIRE 2020.11.28 KOD #4168058

'pyq_executable
[2] C:\Users\deep_bedford\Anaconda3\q\python.q:5: run:py`$pyq_executable
^
[0] ()

)python
'python
[3] python
^
)q
'q
[3] q
^
)

I am not understanding what is wrong now ?

sashkab · 2019-12-02T12:45:01Z

How did you install PyQ? It seems you're not following instructions. As you use Anaconda, you should install it with

conda install -c kx -c enlnt pyq

decranfield1 · 2019-12-02T12:49:28Z

conda install -c kx -c enlnt pyq
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

pyq

Current channels:

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

Somehow conda is not working

github-actions · 2020-02-01T00:07:40Z

Stale issue message

FlyingOE · 2020-03-05T03:34:41Z

q python.q --version

KDB+ 3.6 2019.06.09 Copyright (C) 1993-2019 Kx Systems
w64/ 8(16)core 15794MB user1 user-tp3 10.166.17.120 EXPIRE 2020.07.09 ....

Python 3.7.0

q python.q

KDB+ 3.6 2019.06.09 Copyright (C) 1993-2019 Kx Systems
w64/ 8(16)core 15794MB user1 user-tp3 10.166.17.120 EXPIRE 2020.07.09 ....

Fatal Python error: _Py_InitializeCore: main interpreter already initialized

Current thread 0x0005f078 (most recent call first):

ver

Microsoft Windows [Version 10.0.18363.693]

sashkab · 2020-03-11T00:12:32Z

@FlyingOE please open new issue and provide detailed steps to reproduce this.

abalkin self-assigned this Jul 2, 2015

sashkab added the enhancement label Nov 22, 2016

sashkab mentioned this issue Feb 5, 2017

cann't get QVER #5

Closed

sashkab mentioned this issue Feb 21, 2017

TypeError: 'NoneType' object has no attribute '__getitem__' #8

Closed

abalkin added this to the Release 4.1 milestone Aug 12, 2017

abalkin modified the milestones: Release 4.1, Release 4.1.1 Sep 14, 2017

abalkin mentioned this issue Jul 4, 2018

Multiple CI improvements #73

Merged

github-actions bot added the no-issue-activity label Feb 1, 2020

github-actions bot closed this as completed Feb 8, 2020

perfecto25 mentioned this issue Feb 27, 2020

Pyq on 64-bit Centos 7 failes with Invalid ELF header #128

Closed

7 tasks

Create version of PyQ for Windows? #1

Create version of PyQ for Windows? #1

Comments

rogermcintosh commented Apr 29, 2015

abalkin commented Jul 2, 2015

sashkab commented Aug 30, 2017 • edited by abalkin

Experimental support for Windows has landed with the PyQ 4.1.0.

bevinahal commented Sep 4, 2017 • edited by sashkab

abalkin commented Sep 4, 2017

abalkin commented Sep 14, 2017

sashkab commented Sep 15, 2017 • edited

abalkin commented Sep 15, 2017

ghlian commented Sep 16, 2017 • edited by sashkab

ghlian commented Sep 16, 2017 • edited by sashkab

abalkin commented Sep 16, 2017

abalkin commented Sep 18, 2017 • edited

ghlian commented Sep 18, 2017 • edited by sashkab

abalkin commented Sep 18, 2017

ghlian commented Sep 18, 2017

abalkin commented Sep 19, 2017

ghlian commented Sep 19, 2017 • edited by sashkab

hzadonis commented Sep 19, 2017 • edited by sashkab

abalkin commented Sep 19, 2017

abalkin commented Sep 19, 2017

abalkin commented Sep 19, 2017 • edited

Installing Jupyter kernel

ghlian commented Sep 19, 2017

ghlian commented Sep 19, 2017

abalkin commented Sep 20, 2017

jtzharry commented Aug 9, 2018

sashkab commented Aug 9, 2018

jtzharry commented Aug 10, 2018 • edited

sashkab commented Aug 10, 2018 • edited

jtzharry commented Aug 11, 2018 • edited

sashkab commented Aug 11, 2018 • edited

jtzharry commented Aug 22, 2018

sashkab commented Aug 22, 2018

wjm3333 commented Oct 10, 2018 • edited

sashkab commented Oct 10, 2018

wjm3333 commented Oct 10, 2018

wjm3333 commented Oct 10, 2018 • edited by sashkab

wjm3333 commented Oct 10, 2018 • edited by sashkab

sashkab commented Oct 10, 2018

wjm3333 commented Oct 11, 2018 • edited by sashkab

jmsblah commented Dec 5, 2018

sashkab commented Dec 6, 2018

decranfield1 commented Dec 1, 2019 • edited

sashkab commented Dec 2, 2019

decranfield1 commented Dec 2, 2019 • edited

sashkab commented Dec 2, 2019

decranfield1 commented Dec 2, 2019

github-actions bot commented Feb 1, 2020

FlyingOE commented Mar 5, 2020 • edited

sashkab commented Mar 11, 2020

sashkab commented Aug 30, 2017 •

edited by abalkin

bevinahal commented Sep 4, 2017 •

edited by sashkab

sashkab commented Sep 15, 2017 •

edited

ghlian commented Sep 16, 2017 •

edited by sashkab

ghlian commented Sep 16, 2017 •

edited by sashkab

abalkin commented Sep 18, 2017 •

edited

ghlian commented Sep 18, 2017 •

edited by sashkab

ghlian commented Sep 19, 2017 •

edited by sashkab

hzadonis commented Sep 19, 2017 •

edited by sashkab

abalkin commented Sep 19, 2017 •

edited

jtzharry commented Aug 10, 2018 •

edited

sashkab commented Aug 10, 2018 •

edited

jtzharry commented Aug 11, 2018 •

edited

sashkab commented Aug 11, 2018 •

edited

wjm3333 commented Oct 10, 2018 •

edited

wjm3333 commented Oct 10, 2018 •

edited by sashkab

wjm3333 commented Oct 10, 2018 •

edited by sashkab

wjm3333 commented Oct 11, 2018 •

edited by sashkab

decranfield1 commented Dec 1, 2019 •

edited

decranfield1 commented Dec 2, 2019 •

edited

FlyingOE commented Mar 5, 2020 •

edited