@janeznemanic, how about this?
I got to thinking, and there isn't really a need to check the message contents against the pkgdb acls.
When we run /usr/local/bin/genacls.sh once, it regenerates the local acls cache for every user and every package. So, if the acls don't match for one message or another, it doesn't actually matter for the genacls.sh script itself -- it will just go query pkgdb on its own to get all the correct information.
Furthermore, if we have queued up 16 messages while delaying action, we just want to run genacls.sh once, not 16 times (I think).
No need to actually check message contents.
@janeznemanic could you provide review and comment?
About skipping the check of the message contents. I think you mentioned that genacls.sh requires some time to finish the job, so it is quite heavy workload. So if the consumer receives a lot of 'false' messages with wrong acls, without querying the pkgdb you would run genacls.sh for nothing. But I think that is more theoretical problem which probably doesn't happen in reality. On the other hand it is also true that queries of pkgdb take some time and require resources. So skipping the check of the message content is the right thing to do.
About running the genacls.sh just once instead of 16 times. In my original code you can find these lines in the last for loop:
self.queued_messages = 
The idea behind these lines of code was to achieve exactly what you propose. But I didn't test my code so I don't know if it would work as was intended. I agree running the script once than 16 times is the right step.
Anyway this should get tested and deployed.
Cool. I agree! One thing I don't yet have figured out is this:
This code will run as a plugin to the fedmsg-hub daemon. The fedmsg-hub daemon everywhere runs as the user 'fedmsg'.
However, the genacls cronjob up to this point has been running as a local gen-acls user. I'm not sure of the best way to do this.
Perhaps, we grant the fedmsg user on pkgs01 the right to passwordless sudo to become the gen-acls user? We would then need to modify our subprocess.call(...) to attempt to sudo.
@ralphbean I did some quick research about the problem and here is what I have found out.
Subprocess.call() functions passes all the work to subprocess.Popen() class which enables you to control the creation of new a process more precisely. Here is the Popen's constructor from Python's documentation.
class subprocess.Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
In our case the relevant parameters are args and preexec_fn. Preexec_fn parameter is a function that is called in new process before it is executed. So we need to change the code to something like this.
def change_subprocess_id(user_uid, user_gid):
So this is what I found out. I tested the above solution quickly and it seems to work. More detailed explanation can be found at: http://stackoverflow.com/questions/1770209/run-child-processes-as-different-user-from-a-long-running-process
Cool. @janeznemanic, how about I merge this pull request and we do a separate one for the gid/uid issue.
Would you like to implement it?