Few question from beginners.

[kobzar]

Hello.
First, thanks for the great work.
But I can found answers for my question in doc.
So, may be some one can help me.

1. If I use RedisPool for my own data access in startup parameter like

async def startup(ctx):
    qredis = await arq_create_pool(settings=RedisSettings(host='localhost', port=6379, database=1))
    ctx['redis_cache'] = redis_cache

And use it in my function later

class WorkerSettings:
    functions = [get_messages]
    on_startup = startup
    on_shutdown = shutdown

async def get_messages(ctx):
    redis_cache = ctx['redis_cache']
    print(f"LAST_MAIL_ID: {await redis_cache.get('last_id')}")

Does I am need close this poll in on_shutdown params?

2. How I can run workers from python file? Not from system terminal like
   # arq my_file.WorkeRname

3. How I can work with output log? Disable it or retranslate it in to file?

Thanks

[samuelcolvin]

Do I need to close this poll in on_shutdown

Yes, I think you do need to take care of closing the pool, but I do not know what ill effects occur if you don't.

How can I run the workers from python rather than the terminal?

from arq import Worker

worker_settings = dict(
    functions=[foo, bar], 
    on_startup=startup, 
    on_shutdown=shutdown
)

def run_worker():
    worker = Worker(**worker_settings)
    worker.run()

That's all the CLI is doing, see here.

How can I change the logging setup: disable it or send it to file.

The CLI just calls logging.config.dictConfig with default_log_config.

You'll want to setup your own logging with your own config.

[kobzar]

Hi. You are close issue on GitHub but I couldn’t understand one moment.
I have more that one worker settings in my main file. Something like this

# runner.py
class One:
    functions = [run_one]

class Two:
    functions = [run_two]

So, I’m run two workers for it in terminal

# arq runner.One
# arq runner.Two

After this, I’m run the file #python runner.py

But how I can make that my logs goin to files? Like

arq runner.One >> one.log
arq runner.Two >> two.log

[samuelcolvin]

This is not a question about arq, but rather a general question about python logging. Please read this.

You could use something like this (it won't be perfect as I haven't tested it, but it should be enough to get you going):

import logging.config
import sys

from arq import Worker
from arq.worker import get_kwargs

def log_config(filename: str) -> dict:
    return {
        'version': 1,
        'disable_existing_loggers': False,
        'handlers': {
            'arq.standard': {
                'level': 'INFO', 
                'class': 'logging.FileHandler', 
                'filename': filename, 
                'formatter': 'arq.standard'
            }
        },
        'formatters': {'arq.standard': {'format': '%(asctime)s: %(message)s', 'datefmt': '%H:%M:%S'}},
        'loggers': {'arq': {'handlers': ['arq.standard'], 'level': 'INFO'}},
    }

class One:
    functions = [run_one]

class Two:
    functions = [run_two]

if __name__ == '__main__':
    if sys.argv[1] == 'One':
        settings_cls = One
        filename = 'one.log'
    else:
        settings_cls = Two
        filename = 'two.log'
    logging.config.dictConfig(default_log_config(filename))
    worker = Worker(**get_kwargs(settings_cls))
    worker.run()

[kobzar]

I am added some tests for my workers.
like

async def make_test():
    await asyncio.sleep(1)

for _ in range(0, 100):
    await redis_pool.enqueue_job('make_test')

And now I see on Redis db poll many keys like
arq:result:00b78c85dff3483abe7da58110045e1c = {}

What is it?

[samuelcolvin]

Please open separate issues for separate questions.

Please try to research this stuff yourself before asking questions, I'm not paid to answer questions about open source projects.

It's the key holding the result of jobs, you can change how long the keys live for using settings.

[kobzar]

Very thanks for the support.
I know that it's free soft. But there little information in web and docs.
And some moments are not obvious.
The docs and examples isn't full. But a think a lot of users want to use all the functionality of the library.
maybe update docs and examples will be god idea if will have some of free time to do this.
Sorry one more time.