Pass c_char or c_char_p to ctypes function in numba

[goduck777]

I just realize that numba now does not support ctypes function with argument c_char or c_char_p. It will show an error that this type is not supported.

However, one of the c function I was using requires the first argument to be a single string, "N". Is there a way to get around this limitation and use this C function in jit function?

[stuartarchibald]

Thanks for the request. This is probably something we should think about supporting as it is quite common to want to pass a single char by reference to Fortran libraries.

Whilst not necessarily advisable, below is an example of how we do this sort of thing internally for np.linalg.* support. I've staged a call to BLAS dgemv, passing char "N".

import numpy as np
import ctypes as ct
import sys
import os
from numba import njit

# find the BLAS library, assuming Anaconda NumPy with MKL
the_python = sys.executable
splitter = lambda x: os.path.split(x)[0]
blas_dso = os.path.join(splitter(splitter(the_python)), 'lib', 'libmkl_rt.so')
libblas = ct.CDLL(blas_dso)
assert libblas.dgemv

# bind to BLAS dgemv
DGEMV = libblas.dgemv
double_ty = ct.POINTER(ct.c_double)
int_ty = ct.POINTER(ct.c_int)
# NOTE: the first arg is the `TRANS` argument which is a char by ref, passing it as an `int *`
DGEMV.argtypes = [int_ty, int_ty, int_ty, double_ty, double_ty, int_ty, double_ty, int_ty, double_ty, double_ty, int_ty]
DGEMV.restype=ct.c_void_p

# This is how the character 'N' is going to be passed
TRANS_CHAR = ord('N')

# Stage call in an njit function
@njit
def call_dgemv(alpha, A, x, beta, y):
    m, n = A.shape
    assert len(x) == n
    # pack data into single element arrays so pointers can be grabbed
    alpha_pkd = np.array([alpha], dtype=np.float64)
    beta_pkd = np.array([beta], dtype=np.float64)
    trans_pkd = np.array([TRANS_CHAR], dtype=np.int32)
    inc = np.array([1], dtype=np.int32)
    m_pkd = np.array([m], dtype=np.int32)
    n_pkd = np.array([n], dtype=np.int32)
    # make the call
    DGEMV(trans_pkd.ctypes, m_pkd.ctypes, n_pkd.ctypes, alpha_pkd.ctypes, A.ctypes, m_pkd.ctypes, x.ctypes, inc.ctypes, beta_pkd.ctypes, y.ctypes, inc.ctypes)

# Some test data
m = 5
n = 4
A = np.asfortranarray(np.arange(m * n, dtype=np.float64).reshape(m, n))
x = np.arange(n, dtype=np.float64)
y = np.arange(m, dtype=np.float64)
alpha = 3.
beta = 7.

# make the call
call_dgemv(alpha, A, x, beta, y)
print(y)

# check it matches np equivalent y <- alpha*A*x + beta*y
new_y = np.arange(m, dtype=np.float64)
expected = (alpha*np.dot(A, x) + beta * new_y)

np.testing.assert_allclose(y, expected)

[goduck777]

Thank you. I thought about this idea too. Hope numba can add this feature soon.

[AndrewAnnex]

I would like to bump this issue, I have a python library that relies on ctypes currently and could benefit greatly from supporting both of these.

[rahil-makadia]

Second the bump. I have a complicated pipeline where c_char/c_char_p support is the last step to a desperately needed nopython speedup