New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Singleton service #2444
Singleton service #2444
Conversation
@tardyp, thanks for your PR! By analyzing the history of the files in this pull request, we identified @yetanotherion, @djmitche and @sa2ajj to be potential reviewers. |
eb489bb
to
c64302d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks quite useful
if name in parent.namedServices: | ||
return defer.succeed(parent.namedServices[name]) | ||
try: | ||
o = cls(*args, **kwargs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
o? oh? oh, no!
|
||
@classmethod | ||
def getName(cls, *args, **kwargs): | ||
h = hashlib.sha1() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this PR seems to love one letter names
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
:) h sounds like hash which is a reserved word.
You are right
service.SingletonService.__init__(self) | ||
self.name = self.getName(hyper_host, hyper_accesskey, hyper_secretkey) | ||
# Prepare the parameters for the Docker Client object. | ||
self.client_args = {'clouds': { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to have client_args
public?
}} | ||
|
||
def startService(self): | ||
self.threadPool = threadpool.ThreadPool( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
self.threadPool = threadpool.ThreadPool( | ||
minthreads=1, maxthreads=self.MAX_THREADS, name='hyper') | ||
self.threadPool.start() | ||
self.client = Hyper(self.client_args) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
This class implements a generic Service that needs to be instantiated as a singleton according to its parameters. | ||
It is a common use case to need this for accessing remote services. | ||
Having a singleton to access a remote allows to limit the number of simultaneous access to the same services. | ||
Thus, several completely independent buildbot services can use that singleton to access the service, and automatically synchronize themselves to not overwhelm it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Buildbot
Class method. | ||
Takes same arguments as the constructor of the service. | ||
Get a unique name for that instance of a service. | ||
Used to decide if we need to create a new object. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure I understand what this means.
Takes same arguments as the constructor of the service (plus the `parentService` at the beginning of the list). | ||
Construct an instance of the service if needed, and place it at the beginning of the `parentService` service list. | ||
Placing it at the beginning will guarantee that the singleton will be stopped after the other services. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one empty line here will be enough.
@@ -1,4 +1,3 @@ | |||
# This file is part of Buildbot. Buildbot is free software: you can |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this line is probably removed by accident
o = cls(*args, **kwargs) | ||
except Exception: | ||
return defer.fail(failure.Failure()) | ||
o.name = name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I like this forceful setting of .name
c64302d
to
793a097
Compare
@@ -886,6 +886,44 @@ For example, a particular daily scheduler could be configured on multiple master | |||
Therefore, in this method it is safe to reassign the "active" status to another instance. | |||
This method may return a Deferred. | |||
|
|||
.. py:class:: SingletonService | |||
|
|||
This class implements a generic Service that needs to be instantiated as a singleton according to its parameters. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if you can have n instances of this service but with different parameters this is not a singleton... for accessing to external services, we use normal services "mounted" into a known proxy (with an hardcoded name, which is not very elegant), for instance to be able to have 1 gerrit service talking to Gerrit 2.6 and another talking to Gerrit 2.11.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, this is a parametric singleton, which I argue is very similar concept.
Also, this idea here is that the end-user does not have to know there are singleton to be parametric.
The user will configure a HyperWorker with all the connection parameters in it, and then the HyperWorkers will automatically create the singleton as an implementation detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, but I advice to be very careful in defining it in the class doc for programmers. Of course, users do not care about this detail. But, for most programmers, Singleton
is a design pattern that restricts the instantiation of a class to one object
and
An implementation of the singleton pattern must:
- ensure that only one instance of the singleton class ever exists; and
- provide global access to that instance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you are right, we need to change the name
http://stackoverflow.com/questions/1050991/singleton-with-arguments-in-java
Current coverage is 86.76% (diff: 100%)@@ master #2444 diff @@
==========================================
Files 298 298
Lines 30940 30957 +17
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
+ Hits 26842 26860 +18
+ Misses 4098 4097 -1
Partials 0 0
|
793a097
to
ba100e4
Compare
This is not really a singleton, as this is designed to have multiple instance at the same time.
return defer.succeed(parent.namedServices[name]) | ||
try: | ||
instance = cls(*args, **kwargs) | ||
except Exception: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really want catch all Exception
s here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, we transforms all exceptions into failures
There is one instance of this manager per host, accesskey, secretkey tuple. | ||
This manager manages its own thread pull, as Hyper_sh is blocking | ||
""" | ||
name = "HyperLatentManager" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
quotes
This manager manages its own thread pull, as Hyper_sh is blocking | ||
""" | ||
name = "HyperLatentManager" | ||
MAX_THREADS = 5 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we use this value outside? if not _MAX_THREADS
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, this is eventually designed to be overriden
99f7fa0
to
e3c6b05
Compare
@sa2ajj I think I answered to all the reviews, I am going to merge this as soon as travis is done. |
@tardyp do you really want us to review the change? |
@sa2ajj I do want reviews. I think I addressed everything, and I wanted to move forward, as I have several dependent branches. |
You substantially changed things since the original PR, if I had been the author, I'd wait for the other review, not just Travis results. |
@sa2ajj I just added two trivial comments since your last review, and a note in relnotes There are 800loc to be reviewed in #2446 which are harder to review if the dependent branch is not merged. I am really sorry if I upset you, this was really not my intention. I'm just trying to use the sparse review time the best as I can so that we can move as fast as we can. I have a lot of time now to develop buildbot, and I won't in a few weeks/months, when I'll need to transition to my new paid job (you know how it is). |
This class implements a generic Service that needs to be instantiated as a singleton according to its parameters.
It is a common use case to need this for accessing remote services.
Having a singleton to access a remote allows to limit the number of simultaneous access to the same services.
Thus, several completely independent buildbot services can use that singleton to access the service, and automatically synchronize themselves to not overwhelm it.