Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using @job decorator and func.delay() causes subclass classmethods to not work correctly #2076

Open
amks1 opened this issue Apr 17, 2024 · 2 comments

Comments

@amks1
Copy link
Contributor

amks1 commented Apr 17, 2024

Test code:

# functions_rq.py
import rq.decorators as rqd

class AsyncClassTester():

    def __init__(self, a1, a2) -> None:
        self.a1 = a1
        self.a2 = a2

    @classmethod
    @rqd.job(RQ_HIGH, connection=get_rq_redis_connection())
    def async_classmethod(cls, *args, **kwargs):
        """
        Made to run inside the async worker.
        """
        print(f'Class: {cls.__name__}')
        print(f'Arguments received: {args}')
        print(f'Keyword arguments received: {kwargs}')
        return cls.__name__, args, kwargs
    
class AsyncClassTester_SubClass1(AsyncClassTester):
    pass

class AsyncClassTester_SubClass2(AsyncClassTester):
    pass
# unittest_rq.py
import unittest, rq, time

class RQ_Test(unittest.TestCase):

    def test_subclasses_without_decorator(self):
        # This test passes
        q = rq.Queue(connection=get_rq_redis_connection())
        job2 = q.enqueue(functions_rq.AsyncClassTester_SubClass1.async_classmethod, 'arg1', 'arg2', kwarg1='kwarg1', kwarg2='kwarg2', is_async=RUN_ASYNC)
        time.sleep(5)
        results2 = job2.return_value()
        self.assertEqual(results2[0], 'AsyncClassTester_SubClass1')

    def test_subclasses_with_decorator(self):
        # This test fails 
        job2 = functions_rq.AsyncClassTester_SubClass1.async_classmethod.delay('arg1', 'arg2', kwarg1='kwarg1', kwarg2='kwarg2', is_async=RUN_ASYNC)
        time.sleep(5)
        results2 = job2.return_value()
        self.assertEqual(results2[0], 'AsyncClassTester_SubClass1')

test_subclasses_with_decorator fails with:

AssertionError: 'AsyncClassTester' != 'AsyncClassTester_SubClass1'

Using the @job decorator with delay prevents the correct class from being used inside the job.
However if the method used is an instance method and not a class method, the code runs as expected.

@selwin
Copy link
Collaborator

selwin commented Apr 20, 2024

Any idea on how to fix this?

@terencehonles
Copy link
Contributor

terencehonles commented Aug 28, 2024

This is related to the @classmethod decorator coming after @job, and that the decorator will wrap the function during class construction and when finally using .delay(...) it will internally try to determine the type of the callable, but it will incorrectly deduce it as a function

rq/rq/job.py

Lines 245 to 246 in d5e55ab

elif inspect.isfunction(func) or inspect.isbuiltin(func):
job._func_name = '{0}.{1}'.format(func.__module__, func.__qualname__)
(because that's what it wrapped)

This seems like it would also be a problem for instance methods too, since on class construction they wouldn't be bound yet.

The only way I think you would be able to fix this is to make the @job decorator return a new object implementing the descriptor protocol which would allow getting the object after class creation, but you'd probably also need to make sure it properly emulated any other decorators and those would all have to be applied so the @job decorator sees them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants