feat: parallel limited time load of user META info #71
Conversation
Now Public IP, ASN, Region, ... are loaded using separated helper daemon process Data are shared using shared memory between two processes we have a minimum wait time to let helper process complete, if not ,main process will terminate assuming meta data cannot be found (No Internet or etc)
utils/trace.py
Outdated
| args=(USER_META_INFO_NO_INTERNET, USER_META_INFO_PUBLIC_IP, | ||
| USER_META_INFO_NETWORK_ASN, USER_META_INFO_NETWORK_NAME, | ||
| USER_META_INFO_COUNTRY_CODE, USER_META_INFO_CITY, USER_META_INFO_DONE, user_iface), | ||
| daemon=True).start() |
There was a problem hiding this comment.
It would be cool here if it would be possible to drop privileges for the background process. I am not sure whether this is possible with the Process API. But, if that's the case, I'd really consider doing that. If Process does not support that, a possible way to implement this would be perhaps to drop privileges in get_meta before performing other network-bound operations. I think a good user to drop privileges to is nobody by default and maybe a user should be able to specify the user to drop privileges to (but nobody as the default still looks reasonably good).
Now request is made in serial but lack of timeout is handled using separated process
|
Now everything in |
utils/geolocate.py
Outdated
|
|
||
| def drop_privileges(): | ||
| if os.name == 'posix': | ||
| if os.geteuid() == 0: |
There was a problem hiding this comment.
A long time ago, I have read this paper about dropping privileges: Setuid Demystified.
The most important TL;DR from that paper is to use setresuid and setresgid when available.
In addition to that, I also recommend setting minimal supplementary groups.
More than 10 yeas ago I summarized my understanding of the whole dropping privileges dance in code that I was maintaining at the time. It may be useful to take a look and improve the dropping process.
There was a problem hiding this comment.
Thank you
I agree this is a better approach, I will change the method according to this.
utils/geolocate.py
Outdated
| uid = os.geteuid() | ||
| gid = os.getegid() | ||
| os.setegid(65534) | ||
| os.seteuid(65534) |
There was a problem hiding this comment.
Ah, also: I suggest to use getpwnam to obtain the correct IDs. We cannot assume they're always as such, even though they probably are. We would like to ensure we're using the right IDs.
There was a problem hiding this comment.
We did so until 2d1dee9 as it requires loading libraries (pwd & grp) I checked the current state of BSD & Linux and it seems they are using overflowuid (from /proc/sys/fs/overflowuid ) as the noboy and nogroup id, so we thought we can reduce imported modules by hard-coding this id as it seems in current 64bit systems it is always 65534
we use independent process in posix system to make privilege dropping stable and thread based approach in windows to prevent windows system get suspicious and block requests
we use independent process in posix system to make privilege dropping stable and thread based approach in windows to prevent windows system get suspicious and block requests
move unix privilege dropping into process itself
RYNEQ
force-pushed
the
feat/parallel-metadata-query
branch
from
a4fc56b
to
bf5c82a
Compare
Co-authored-by: Simone Basso <bassosimone@gmail.com>
Co-authored-by: Simone Basso <bassosimone@gmail.com>
|
there is a race condition that should be fixed. but it's a UX bug, so we are good for now and will take care of it later. |
Now Public IP, ASN, Region, ... are loaded using separated helper daemon process
Data are shared using shared memory between two processes
we have a minimum wait time to let helper process complete, if not ,main
process will terminate assuming meta data cannot be found (No Internet or etc)