Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Portus freezes when trying to get manifest after image push #373
We use Portus in the docker environment. Both our registry as well as Portus use TLS. We do not use any proxy in addition to that set-up. The code we use is the current (15/09/30) master from github.
Whenever we try to push an image, Portus freezes exactly at the function get_response_token, line 140
If we change the function a little bit and add
Even more strange, when we call the manifest function from the rails console:
... everything works as excepted.
This problem was first mentioned in another issue: #338
referenced this issue
Sep 30, 2015
I have additional information. I added set_debug_output to the request. It seems, that the connection is established, however, http.start does not return. The following output is generated:
Ok, after debugging the problem, it seems like it's quite a straight-forward problem and it's not Portus' fault really. I was using a config in which Portus was behind NGinx with SSL and my registry run on its own with SSL too. The thing is that when Portus gets a notification, it will make a GET request to the registry in order to get the tag that has been pushed (we didn't do that before distribution v2.1, but it's required now). Anyways, in order to do that, you need a free SSL connection in place, and in this setup you don't, because it's already busy by the registry -> Portus web event. Thus, in this case Portus didn't hang in an infinite loop, rather it waited for the connection to be free. In my case I had to wait until the connection timed out, and then Portus could fetch the manifest. Even in this case, the tag would not be pushed into Portus' DB because the first connection timed out, and therefore errored.
The solution, therefore, is to either use the passenger config that we are using in the rpm that we build (and if I recall correctly, we didn't write any specific config for passenger to work here, it works out of the box :D), or to come up with a configuration for NGinx or another that respects that. For example, an approach that worked for my NGinx config was the one described here. Basically in this config you'll end up using two sockets with Thin, instead of one single connection.
Since this is tricky, I'll add a page on the wiki about deploying Portus.
Please, tell me if this solution works for you, and thanks for your patience :)
Update: I've already added it on the wiki: https://github.com/SUSE/Portus/wiki/Installing-Portus#known-issues
Thanks for the excellent work solving this problem. I can confirm - the problem is solved for us. We can now successfully push an image to the registry and all information are gathered without Portus freezing.
The configuration for running this in docker is now quite extensive. Shall i document it somewhere or send it to you so that you can have a look what of the configuration settings seems relevant for other users?
I documented our configuration on https://github.com/morodin/Portus/wiki/Portus-SSL-Docker-Configuration. I really appreciate all comments or correction.