How do I casbin_redis_watcher with a device?

[YihuiLu]

I throw an error when I use the following code

def add_authentication_middleware(app: FastAPI):
    class BasicAuth(AuthenticationBackend):
        async def authenticate(self, request):
            if "Authorization" not in request.headers:
                return None

            auth = request.headers["Authorization"]
            try:
                scheme, credentials = auth.split()
                decoded = base64.b64decode(credentials).decode("ascii")
            except (ValueError, UnicodeDecodeError, binascii.Error):
                raise AuthenticationError("Invalid basic auth credentials")

            username, _, password = decoded.partition(":")
            return AuthCredentials(["authenticated"]), SimpleUser(username)

    model_file_path = Path(__file__).parent / "core/configs/casbin_model.conf"
    model_file_path = str(model_file_path.absolute())

    adapter = casbin_sqlalchemy_adapter.Adapter(settings.SQLALCHEMY_DATABASE_URI)

    enforcer = casbin.Enforcer(model_file_path, adapter, settings.CASBIN_LOG)

    watcher = RedisWatcher(
        settings.CASBIN_REDIS_WATCHER_HOST, settings.CASBIN_REDIS_WATCHER_PORT
    )
    watcher.set_update_callback(enforcer.load_policy)
    enforcer.set_watcher(watcher)

    app.add_middleware(CasbinMiddleware, enforcer=enforcer)
    app.add_middleware(AuthenticationMiddleware, backend=BasicAuth())

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/main.py", line 4, in <module>
    app = create_app()
  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/app/__init__.py", line 75, in create_app
    add_authentication_middleware(app)
  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/app/__init__.py", line 107, in add_authentication_middleware
    settings.CASBIN_REDIS_WATCHER_HOST, settings.CASBIN_REDIS_WATCHER_PORT
  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/.venv/lib/python3.7/site-packages/casbin_redis_watcher/watcher.py", line 44, in __init__
    self.create_subscriber_process(start_process)
  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/.venv/lib/python3.7/site-packages/casbin_redis_watcher/watcher.py", line 52, in create_subscriber_process
    p.start()
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/Users/luyifei/.pyenv/versions/3.7.9/lib/python3.7/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

[hsluoyz]

@Zxilly

[Zxilly]

working on this

[Zxilly]

I noticed you use a package named casbin_redis_watcher, but I didn't find this on pypi. Could you please provide the link of that? This will be helpful to locate the problem.

Seems the problem caused by

  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/.venv/lib/python3.7/site-packages/casbin_redis_watcher/watcher.py", line 44, in __init__
    self.create_subscriber_process(start_process)

Maybe you should setup watcher and enforcer in the __main__

[Zxilly]

I guess it use popen to operate redis, which may cause unexpected error when it works with fastapi

[YihuiLu]

我注意到您使用了名为的程序包casbin_redis_watcher，但是我在pypi上没有找到它。您能提供它的链接吗？这将有助于定位问题。

似乎是由问题引起的

  File "/Users/luyifei/PycharmProjects/pinstreet_open_service_backend/.venv/lib/python3.7/site-packages/casbin_redis_watcher/watcher.py", line 44, in __init__
    self.create_subscriber_process(start_process)

也许您应该在 __main__

casbin_redis_watcher's home page is https://pypi.org/project/pycasbin-redis-watcher/ hope it's

casbin_redis_watcher is an officially recommended plug-in for casbin to synchronize permission descriptions for different nodes This is necessary for me

[Zxilly]

@YihuiLu
As described in redis-watcher readme

This redis-watcher module starts separate processes which subscribe to a redis channel, and listens for updates to the casbin policy on that channel. When running within WSGI contexts (like uwsgi) you may want to start these processes as a postfork action.

So, you should use startup action for fastapi.
ref to: https://fastapi.tiangolo.com/advanced/events/

[YihuiLu]

@YihuiLu
As described in redis-watcher readme

This redis-watcher module starts separate processes which subscribe to a redis channel, and listens for updates to the casbin policy on that channel. When running within WSGI contexts (like uwsgi) you may want to start these processes as a postfork action.

So, you should use startup action for fastapi.
ref to: https://fastapi.tiangolo.com/advanced/events/

Thank you for your reply, I've tried to use casbin_redis_watcher in startup, but there's another problem: ScienceLogic/pycasbin-redis-watcher#2
I know you shouldn't be asked questions that have nothing to do with fastapi-authz at this time, but you'll need these two frameworks to work with when using casbin in fastapi, so I'm looking forward to getting your help. How do I design a code structure to run successfully?

[Zxilly]

Try something like

proc = None


def run(): 
    uvicorn.run(app=app, host=host, port=port)


def start():
    global proc
    proc = Process(target=run, args=(), daemon=True)
    proc.start()

and use start() in the __main__

[Zxilly]

In fact, this is still not a good idea. pycasbin-redis-watcher looks not designed for asyncio.
Maybe later I will create a asyncio version.

[YihuiLu]

In fact, this is still not a idea. pycasbin-redis-watcher looks not designed for asyncio.
Maybe later I will create a asyncio version.

I look forward to your writing of an asyncio version, and I hope to be able to help you.

For the time being, I might consider giving up Casbin and Fastapi for the time being, because it's more difficult to

But if I can set a plan for writing an asyncio version of Watcher, I'm willing to participate and make it part of my job