Fix IP overlap with empty EndpointSpec #2505
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2505 +/- ##
==========================================
+ Coverage 61.32% 61.68% +0.36%
==========================================
Files 131 129 -2
Lines 21451 21293 -158
==========================================
- Hits 13154 13135 -19
+ Misses 6878 6753 -125
+ Partials 1419 1405 -14 |
@@ -940,7 +940,7 @@ func updatePortsInHostPublishMode(s *api.Service) { | |||
s.Endpoint.Spec = s.Spec.Endpoint.Copy() | |||
} | |||
|
|||
func (a *Allocator) allocateService(ctx context.Context, s *api.Service) error { | |||
func (a *Allocator) allocateService(ctx context.Context, s *api.Service, restart bool) error { |
anshulpundir
Feb 6, 2018
Contributor
Please add a comment for this function along with a description for each of the params.
Please add a comment for this function along with a description for each of the params.
fcrisciani
Feb 7, 2018
Author
done
done
@@ -306,6 +306,7 @@ func validateTaskSpec(taskSpec api.TaskSpec) error { | |||
func validateEndpointSpec(epSpec *api.EndpointSpec) error { | |||
// Endpoint spec is optional | |||
if epSpec == nil { | |||
epSpec = &api.EndpointSpec{Mode: api.ResolutionModeVirtualIP} |
anshulpundir
Feb 6, 2018
Contributor
I would suggest doing this outside this function. This function is for validation purposes only and should not have any side effects.
I would suggest doing this outside this function. This function is for validation purposes only and should not have any side effects.
dperny
Feb 6, 2018
Collaborator
I strongly agree. We should not modify the user's spec after it's been sent to Swarmkit. There are Good Reasons to do this.
I strongly agree. We should not modify the user's spec after it's been sent to Swarmkit. There are Good Reasons to do this.
fcrisciani
Feb 7, 2018
Author
moved it outside
moved it outside
anshulpundir
Feb 7, 2018
Contributor
also probably return an InvalidArgument here.
also probably return an InvalidArgument here.
fcrisciani
Feb 7, 2018
Author
then we don't need to set any default if we bail out with an error
then we don't need to set any default if we bail out with an error
anshulpundir
Feb 7, 2018
Contributor
The change essentially is to make EndpointSpec mandatory, correct ? So, to avoid breaking old clients, we're populating it in swarm.
Its OK to return an error from validateEndpointSpec and handle it wherever it is called by filling in an empty EndpointSpec.
The change essentially is to make EndpointSpec mandatory, correct ? So, to avoid breaking old clients, we're populating it in swarm.
Its OK to return an error from validateEndpointSpec and handle it wherever it is called by filling in an empty EndpointSpec.
@@ -1188,7 +1190,7 @@ func (a *Allocator) procUnallocatedServices(ctx context.Context) { | |||
var allocatedServices []*api.Service | |||
for _, s := range nc.unallocatedServices { | |||
if !nc.nwkAllocator.IsServiceAllocated(s) { | |||
if err := a.allocateService(ctx, s); err != nil { | |||
if err := a.allocateService(ctx, s, false); err != nil { |
anshulpundir
Feb 6, 2018
Contributor
nit: add a comment on why restart flag is false here.
nit: add a comment on why restart flag is false here.
@@ -587,8 +587,8 @@ func (a *Allocator) allocateServices(ctx context.Context, existingAddressesOnly | |||
continue | |||
} | |||
|
|||
if err := a.allocateService(ctx, s); err != nil { | |||
log.G(ctx).WithError(err).Errorf("failed allocating service %s during init", s.ID) | |||
if err := a.allocateService(ctx, s, existingAddressesOnly); err != nil { |
anshulpundir
Feb 6, 2018
Contributor
nit: Please add a comment on how existingAddressesOnly relates to the restart flag which is an arg for allocateService()
nit: Please add a comment on how existingAddressesOnly relates to the restart flag which is an arg for allocateService()
fcrisciani
Feb 7, 2018
Author
change the name of the variable to match to show that the relation is that they are the same thing
change the name of the variable to match to show that the relation is that they are the same thing
fcrisciani
Feb 7, 2018
Author
the name of the variable is changed inside the allocateService not here, I kept the original name existingAddressesOnly
the name of the variable is changed inside the allocateService not here, I kept the original name existingAddressesOnly
@@ -274,7 +274,7 @@ func (a *Allocator) doNetworkAlloc(ctx context.Context, ev events.Event) { | |||
} | |||
updatePortsInHostPublishMode(s) | |||
} else { | |||
if err := a.allocateService(ctx, s); err != nil { | |||
if err := a.allocateService(ctx, s, false); err != nil { |
anshulpundir
Feb 6, 2018
Contributor
nit: add a comment on why restart flag is false here.
nit: add a comment on why restart flag is false here.
@@ -244,7 +244,7 @@ func (a *Allocator) doNetworkAlloc(ctx context.Context, ev events.Event) { | |||
break | |||
} | |||
|
|||
if err := a.allocateService(ctx, s); err != nil { | |||
if err := a.allocateService(ctx, s, false); err != nil { |
anshulpundir
Feb 6, 2018
Contributor
nit: add a comment on why restart flag is false here.
nit: add a comment on why restart flag is false here.
fcrisciani
Feb 7, 2018
Author
I put a comment on the allocateService itself, instead to spread a one line everywhere saying that is not a restart case
I put a comment on the allocateService itself, instead to spread a one line everywhere saying that is not a restart case
@@ -244,6 +245,7 @@ vipLoop: | |||
} | |||
for _, nAttach := range specNetworks { | |||
if nAttach.Target == eAttach.NetworkID { | |||
log.L.WithFields(logrus.Fields{"service_id": s.ID, "vip": eAttach.Addr}).Infof("allocate vip") |
anshulpundir
Feb 6, 2018
Contributor
super nit: use Info instead fo Infof ?
super nit: use Info instead fo Infof ?
abhi
Feb 6, 2018
Contributor
I think this should be Debug ? Info will print it for every ip ?
I think this should be Debug ? Info will print it for every ip ?
fcrisciani
Feb 6, 2018
Author
I was actually thinking to want this printed. Concerns?
I was actually thinking to want this printed. Concerns?
nishanttotla
Feb 6, 2018
•
Contributor
@fcrisciani only concern is that it might be too frequent to print in the logs. We normally would like to avoid that.
@fcrisciani only concern is that it might be too frequent to print in the logs. We normally would like to avoid that.
dperny
Feb 6, 2018
Collaborator
Definitely change from Info
to Debug
. It'll be too verbose for Info
.
Definitely change from Info
to Debug
. It'll be too verbose for Info
.
fcrisciani
Feb 6, 2018
Author
@nishanttotla today we have 0 visibility and debug is very difficult, do you have any other idea?
@nishanttotla today we have 0 visibility and debug is very difficult, do you have any other idea?
fcrisciani
Feb 7, 2018
Author
Guys putting it to debug, but we will continue to have no clue of what is going on and what is assigned especially when allocation are not correct
Guys putting it to debug, but we will continue to have no clue of what is going on and what is assigned especially when allocation are not correct
@@ -796,6 +796,145 @@ func TestAllocatorRestoreForDuplicateIPs(t *testing.T) { | |||
} | |||
} | |||
|
|||
func TestAllocatorRestartNoEndpointSpec(t *testing.T) { |
anshulpundir
Feb 6, 2018
Contributor
Please add a summery of this test case.
Please add a summery of this test case.
fcrisciani
Feb 7, 2018
Author
done
done
LGTM. One nit |
@@ -244,6 +245,7 @@ vipLoop: | |||
} | |||
for _, nAttach := range specNetworks { | |||
if nAttach.Target == eAttach.NetworkID { | |||
log.L.WithFields(logrus.Fields{"service_id": s.ID, "vip": eAttach.Addr}).Infof("allocate vip") |
abhi
Feb 6, 2018
Contributor
I think this should be Debug ? Info will print it for every ip ?
I think this should be Debug ? Info will print it for every ip ?
if err := a.allocateService(ctx, s); err != nil { | ||
log.G(ctx).WithError(err).Errorf("failed allocating service %s during init", s.ID) | ||
if err := a.allocateService(ctx, s, existingAddressesOnly); err != nil { | ||
log.G(ctx).WithField("existingAddressesOnly", existingAddressesOnly).WithError(err).Errorf("failed allocating service %s during init", s.ID) |
nishanttotla
Feb 6, 2018
Contributor
nit: can this comment be made more clear? Should be it something like "failed to allocate network..." or something of that nature, perhaps with more info?
nit: can this comment be made more clear? Should be it something like "failed to allocate network..." or something of that nature, perhaps with more info?
dperny
Feb 6, 2018
Collaborator
The error field will give that information.
The error field will give that information.
9be2619
to
03eda38
LGTM |
Looks good. Minor nits. I'll merge as soon as you address these. |
@@ -796,6 +796,149 @@ func TestAllocatorRestoreForDuplicateIPs(t *testing.T) { | |||
} | |||
} | |||
|
|||
// TestAllocatorRestartNoEndpointSpec covers the leader election case when the service Spec | |||
// does not contains the EndpointSpec. |
anshulpundir
Feb 7, 2018
Contributor
nit: does not contains => does not contain
nit: does not contains => does not contain
fcrisciani
Feb 7, 2018
Author
done
done
@@ -796,6 +796,149 @@ func TestAllocatorRestoreForDuplicateIPs(t *testing.T) { | |||
} | |||
} | |||
|
|||
// TestAllocatorRestartNoEndpointSpec covers the leader election case when the service Spec | |||
// does not contains the EndpointSpec. | |||
// The expected behavior iis that the VIP(s) are still correctly populated inside |
anshulpundir
Feb 7, 2018
Contributor
iis => is
iis => is
fcrisciani
Feb 7, 2018
Author
done
done
@@ -587,8 +587,8 @@ func (a *Allocator) allocateServices(ctx context.Context, existingAddressesOnly | |||
continue | |||
} | |||
|
|||
if err := a.allocateService(ctx, s); err != nil { | |||
log.G(ctx).WithError(err).Errorf("failed allocating service %s during init", s.ID) | |||
if err := a.allocateService(ctx, s, existingAddressesOnly); err != nil { |
@@ -940,7 +940,10 @@ func updatePortsInHostPublishMode(s *api.Service) { | |||
s.Endpoint.Spec = s.Spec.Endpoint.Copy() | |||
} | |||
|
|||
func (a *Allocator) allocateService(ctx context.Context, s *api.Service) error { | |||
// allocateService takes care to align the desired state with the spec passed | |||
// the last parameter is true only during restart when the data are read from raft |
anshulpundir
Feb 7, 2018
Contributor
nit: data are => data is
nit: data are => data is
fcrisciani
Feb 7, 2018
Author
done
done
Passing and empty EndpointSpec in the service spec was correctly triggering the VIP allocation but the leader election was erroneusly handling the IPAM state restore trying to release the VIP. The fix focuses on proper handling of the restart case. Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
Passing and empty EndpointSpec in the service spec
was correctly triggering the VIP allocation but
the leader election was erroneusly handling the IPAM
state restore.
This fix ensure that the EndpointSpec if not specified is
actually added to the ServiceSpec selection endpoint mode VIP.
Also the allocate service has now the restart flag that will skip
the deallocation logic that was erroneously triggered.
Fix: moby/moby#33795
Signed-off-by: Flavio Crisciani flavio.crisciani@docker.com