Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can Pyston run uvicorn server? #76

Closed
EmadMokhtar opened this issue Aug 8, 2021 · 8 comments
Closed

Can Pyston run uvicorn server? #76

EmadMokhtar opened this issue Aug 8, 2021 · 8 comments

Comments

@EmadMokhtar
Copy link

I'm building a Web API using FastAPI framework. It used uvicorn as a server to run the application. Can Pyston run uvicorn server? I'm not sure because I'm getting this error.

uvicorn main:app --reload                                                                                        
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [31274] using watchgod
Traceback (most recent call last):
  File "/home/emadmokhtar/.pyenv/versions/fastapi-pyston/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/main.py", line 371, in main
    run(app, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/main.py", line 388, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/supervisors/basereload.py", line 43, in run
    self.startup()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/supervisors/basereload.py", line 64, in startup
    self.process.start()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_spawn_posix.py", line 39, in _launch
    from . import resource_tracker
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/resource_tracker.py", line 38, in <module>
    import _posixshmem
ImportError: /home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/lib-dynload/../../../lib/librt.so.1: undefined symbol: __clock_nanosleep, version GLIBC_PRIVATE
@EmadMokhtar
Copy link
Author

It can run but without the --reload flag

uvicorn main:app
INFO:     Started server process [31373]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

@kmod
Copy link
Contributor

kmod commented Aug 11, 2021

It looks like this is via pyenv which uses our portable release, and it looks like sometimes our portable release is not fully portable and has issues like this. @undingen knows more but he's on vacation

In the mean time, you could try our .deb releases which are built for a specific distro and shouldn't have this sort of issue. This would also let you try our new 2.3 release which includes a number of compatibility improvements which could potentially help with a library like this.

@undingen
Copy link
Contributor

Sorry for the delay, just came back from holiday.
I can confirm your issue and it's caused by mixing the distro shared libraries with the one included in the portable release used by pyenv. It seems like the portable release comes with a bunch of pitfalls and needs to overhaul. In the meantime either using the debian package or removing/renaming the included librt.so should fix the problem:

mv /home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/lib-dynload/../../../lib/librt.so.1 /home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/lib-dynload/../../../lib/librt.so.1_backup

undingen added a commit that referenced this issue Sep 2, 2021
@undingen
portable release: prefer distro supplied shared objects
ec6f7ef 
Until now we always used our supplied 3rd party shared objects dependencies over
the distro supplied ones. But ours are likely older which causes problems.
E.g. issue 41: we load a thirdparty distro library compiled against a newer zlib but it's loading
our supplied older version which fails.

This patch changes the search order to use our supplied .so's only as fallback if the dist ones are not available.

This is noticable e.g. running `pyston -c "import sqlite3; print(sqlite3.sqlite_version)"`
will output 3.31.1 (on my ubuntu 20.04 system) instead of 3.11.0.

Why do we even supply this libraries?:
We do have to supply this libraries in case the user does not have the library installed
or it's not available in the required major version. E.g. if you use the portable release on latest fedora and ubuntu
openssl 1.0.x is not available anymore (it uses 1.1.x) so pip will not work if we don't supply 1.0.
Which means in order to have only one release which works on as much distros as possible we need to supply this
files but should make sure they are only used as fallback.

I verified that this fixes the problems in #41 and #76 on ubuntu, fedora and archlinux.

BTW: patchelf v0.10 supplied in ubuntu 20.04 contains a bug which corrupts files when modifying
already previously via patchelf modified executables.
@undingen undingen mentioned this issue Sep 2, 2021
Merged
@undingen
Copy link
Contributor

undingen commented Sep 7, 2021

I recreated the portable release with the #83 fix applied which fixes your problem - you can download it here:
https://github.com/pyston/pyston/releases/download/v2.3/pyston_2.3_portable-v3.tar.gz

@undingen undingen closed this as completed Sep 7, 2021
@JohnMoutafis
Copy link

Ran on the same issue using pyston portable v.2.3.2 trying to start a FastAPI application via uvicorn:

uvicorn apis.main:app --workers 4 --reload --debug

@undingen
Copy link
Contributor

undingen commented Mar 3, 2022

What distro are you using? It seemed to work when I tried it out on ubuntu 20.04

@JohnMoutafis
Copy link

@undingen I am using Linux Mint 20.2

@undingen
Copy link
Contributor

undingen commented Mar 3, 2022

I seem not not be able to reproduce it using linuxmintd/mint20.2-amd64 docker and the example from https://www.uvicorn.org/ using your command line args.
Could you please try reproduce/reduce it and and open a new bug.
Thanks

@Azulinho Azulinho mentioned this issue Sep 11, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@kmod @EmadMokhtar @JohnMoutafis @undingen