Fix for string tests failed due to bad str_overload

[kozlov-alexey]

No description provided.

[fschlimb]

This looks good.
Please provide a sentence or two per loop in a comment telling what it does/how it works.

[shssf]

Do you think it is necessary to implement several function via auto generated source?
I would vote to make them as regular functions with required decorators.
@fschlimb What do you think about it? I mean, it might be better to stay away from such technique of programming.

[fschlimb]

I usually try to avoid code duplication as much as possible. There will be cases where such code generation is required. So I tend to think it is not too bad.
Even though I am not sure I would now to do this with only decorators, such simpler cases can be handled with higher lever functions which call @overload. Here is a small POC:

from numba import njit, objmode, types
from numba.extending import overload

# dummy default python funcs
def x(a):
    return a+a
def y(a):
    return a*a
def z(a):
    return a**a

# a function generically overloading a given function with one arg
def ovl(func):
    @overload(func)
    def _f(a):
        def _ff(a):
            with objmode(r='int32'):
                r = func(a)
                print(':) -> ', func(a))
            return r
        return _ff

# use the above overloader to overload x and y
ovl(x)
ovl(y)

# we can now use them in a jitted function
@njit
def z():
    return x(4) + y(4)

r = z()
print('24 ?=', r)

I am not sure which one I like better. I am torn.

[shssf]

@fschlimb I agree, code duplication is not good approach but autogeneration is also not good (I would say it is worse than code duplication). I would vote for any implementation that avoid code generation.

[fschlimb]

fine with me as long as we are not duplicating code.
Just out of curiosity: why do you think code generation is bad? HPAT is a code-generator, numba is, and even python is. So this is not really unusual in this context.

[shssf]

@fschlimb Because it non debuggable, non performance assessment, and etc. It only looks like a good approach but if something goes wrong it will be a real headache.

[shssf]

@fschlimb We have one more option - generate the code at build stage, but avoid doing this in runtime.

[kozlov-alexey]

Frank, Sergey, thank you both for provided input!
I will try to re-write it using provided example, as to me it looks better than calling exec (both due to debugging and security considerations).

[shssf]

@kozlov-alexey Do you plan to make changes in this PR?

[kozlov-alexey]

@shssf Yes, I'm about to push it)

[kozlov-alexey]

As per suggestion above I've rewritten it using decorators and getitem.
Two issues are observed:

1. Test now runs a bit slower, 4.6-4.7 seconds vs 4.2-4.3 seconds before (not sure why it takes more time), that needs to be investigated probably.
2. Wasn't able to verify on master due to import error from this line in test_strings.py:
   "from .gen_test_data import ParquetGenerator" (even after installing pyspark). I was able to verify only based on the commit prior to [BUG] Fixed problems with generation parquet files #93.

[shssf]

2. Wasn't able to verify on master due to import error from this line in test_strings.py:
   "from .gen_test_data import ParquetGenerator" (even after installing pyspark). I was able to verify only based on the commit prior to [BUG] Fixed problems with generation parquet files #93.

PR #96 might help