Skip to content

removed lock for version command #929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 12, 2021
Merged

removed lock for version command #929

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
36b04ff
removed lock for version command
tamilmani1989 Jul 10, 2021
1651765
updated variable name
tamilmani1989 Jul 10, 2021
96c9ba9
catch and return error on lock
tamilmani1989 Jul 12, 2021
0a802d9
added log for release lock file
tamilmani1989 Jul 12, 2021
91bf31d
separated out lock and store initialization
tamilmani1989 Jul 12, 2021
a342c91
addressed comments
tamilmani1989 Jul 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 88 additions & 7 deletions cni/network/network.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ const (
ipVersion = "4"
ipamV6 = "azure-vnet-ipamv6"
defaultRequestTimeout = 15 * time.Second
azureCniName = "azure-vnet"
)

// CNI Operation Types
Expand Down Expand Up @@ -145,12 +146,6 @@ func (plugin *netPlugin) Start(config *common.PluginConfig) error {
common.LogNetworkInterfaces()

// Initialize network manager.
err = plugin.nm.Initialize(config, rehydrateNetworkInfoOnReboot)
Copy link
Member

@neaggarwMS neaggarwMS Jul 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we move this? #Resolved

Copy link
Member Author

@tamilmani1989 tamilmani1989 Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we restore state inside this call which we have to do after lock

if err != nil {
log.Printf("[cni-net] Failed to initialize network manager, err:%v.", err)
return err
}

log.Printf("[cni-net] Plugin started.")

return nil
Expand Down Expand Up @@ -282,7 +277,6 @@ func (plugin *netPlugin) setCNIReportDetails(nwCfg *cni.NetworkConfig, opType st
plugin.report.SubContext = fmt.Sprintf("%+v", nwCfg)
plugin.report.EventMessage = msg
plugin.report.BridgeDetails.NetworkMode = nwCfg.Mode
plugin.report.InterfaceDetails.SecondaryCAUsedCount = plugin.nm.GetNumberOfEndpoints("", nwCfg.Name)
}

func addNatIPV6SubnetInfo(nwCfg *cni.NetworkConfig,
Expand All @@ -301,6 +295,33 @@ func addNatIPV6SubnetInfo(nwCfg *cni.NetworkConfig,
}
}

func acquireLockForStore(config *common.PluginConfig, plugin *netPlugin) error {
var err error
if err = plugin.Store.Lock(true); err != nil {
Comment on lines +299 to +300
Copy link
Collaborator

@rbtr rbtr Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

following this err is hard. it might be better to make this a one-line declaration/initialization (if err := ...; err != nil) and in the nested ifs below, explicitly return err or nil early instead of sometimes setting err = nil and returning err at the end of the function.

Copy link
Member Author

@tamilmani1989 tamilmani1989 Jul 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wanted to avoid multiple returns but i agree it affects readability. Let me redesign this a bit

Copy link
Collaborator

@rbtr rbtr Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple returns are okay, in go it's encouraged to return early when you can. returning early on errors lets us keep the code linear and not need to nest it as deeply.

log.Printf("[CNI] Failed to lock store: %v. check if process running", err)
if isSafe, _ := plugin.IsSafeToRemoveLock(azureCniName); isSafe {
log.Printf("[CNI] Removing lock file as process holding lock exited")
if err = releaseLockForStore(plugin); err != nil {
log.Errorf("Failed to release lock file, err:%v.\n", err)
}
}
}

return err
}

func releaseLockForStore(plugin *netPlugin) error {
if plugin.Store != nil {
err := plugin.Store.Unlock(false)
if err != nil {
log.Printf("[cni] Failed to unlock store: %v.", err)
return err
}
}

log.Printf("Released lock file")
return nil
}
//
// CNI implementation
// https://github.com/containernetworking/cni/blob/master/SPEC.md
Expand Down Expand Up @@ -349,6 +370,22 @@ func (plugin *netPlugin) Add(args *cniSkel.CmdArgs) error {
iptables.DisableIPTableLock = nwCfg.DisableIPTableLock
plugin.setCNIReportDetails(nwCfg, CNI_ADD, "")

// acquire cni lock file
// TODO: check if we need pluginconfig and if not remove it
config := &common.PluginConfig{Store: plugin.Store}
if err := acquireLockForStore(config, plugin); err != nil {
log.Errorf("Couldn't acquire lock: %+v", err)
return err
}

defer releaseLockForStore(plugin)
Copy link
Member

@neaggarwMS neaggarwMS Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defer releaseLockForStore(plugin)

Shouldnt this be above acquire, always safe to release in case it got acquired above and failed after

Copy link
Member Author

@tamilmani1989 tamilmani1989 Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope not needed, if this fails no need to unlock


// restore network state
if err = plugin.nm.Initialize(config, rehydrateNetworkInfoOnReboot); err != nil {
log.Printf("[cni-net] Failed to initialize network manager, err:%+v.", err)
return err
}

defer func() {
operationTimeMs := time.Since(startTime).Milliseconds()
cniMetric.Metric = aitelemetry.Metric{
Expand Down Expand Up @@ -774,6 +811,20 @@ func (plugin *netPlugin) Get(args *cniSkel.CmdArgs) error {

iptables.DisableIPTableLock = nwCfg.DisableIPTableLock

config := &common.PluginConfig{Store: plugin.Store}
if err := acquireLockForStore(config, plugin); err != nil {
log.Errorf("Couldn't acquire lock: %+v", err)
return err
}

defer releaseLockForStore(plugin)

// restore network state
if err = plugin.nm.Initialize(config, rehydrateNetworkInfoOnReboot); err != nil {
log.Printf("[cni-net] Failed to initialize network manager, err:%+v.", err)
return err
}

// Parse Pod arguments.
if k8sPodName, k8sNamespace, err = plugin.getPodInfo(args.Args); err != nil {
return err
Expand Down Expand Up @@ -859,6 +910,21 @@ func (plugin *netPlugin) Delete(args *cniSkel.CmdArgs) error {

log.Printf("[cni-net] Read network configuration %+v.", nwCfg)

// acquire cni lock file
config := &common.PluginConfig{Store: plugin.Store}
if err := acquireLockForStore(config, plugin); err != nil {
log.Errorf("Couldn't acquire lock: %+v", err)
return err
}

defer releaseLockForStore(plugin)

// restore network state
if err = plugin.nm.Initialize(config, rehydrateNetworkInfoOnReboot); err != nil {
log.Printf("[cni-net] Failed to initialize network manager, err:%+v.", err)
return err
}

// Parse Pod arguments.
if k8sPodName, k8sNamespace, err = plugin.getPodInfo(args.Args); err != nil {
log.Printf("[cni-net] Failed to get POD info due to error: %v", err)
Expand Down Expand Up @@ -1021,6 +1087,21 @@ func (plugin *netPlugin) Update(args *cniSkel.CmdArgs) error {
iptables.DisableIPTableLock = nwCfg.DisableIPTableLock
plugin.setCNIReportDetails(nwCfg, CNI_UPDATE, "")

// acquire cni lock file
config := &common.PluginConfig{Store: plugin.Store}
if err := acquireLockForStore(config, plugin); err != nil {
log.Errorf("Couldn't acquire lock: %+v", err)
return err
}

defer releaseLockForStore(plugin)

// restore network state
if err = plugin.nm.Initialize(config, rehydrateNetworkInfoOnReboot); err != nil {
log.Printf("[cni-net] Failed to initialize network manager, err:%+v.", err)
return err
}

defer func() {
operationTimeMs := time.Since(startTime).Milliseconds()
cniMetric.Metric = aitelemetry.Metric{
Expand Down
10 changes: 1 addition & 9 deletions cni/network/plugin/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,8 +180,7 @@ func main() {
log.Printf("Failed to create network plugin, err:%v.\n", err)
return
}

// CNI Acquires lock
// CNI initializes store
if err = netPlugin.Plugin.InitializeKeyValueStore(&config); err != nil {
log.Errorf("Failed to initialize key-value store of network plugin, err:%v.\n", err)
tb := telemetry.NewTelemetryBuffer()
Expand All @@ -190,13 +189,6 @@ func main() {
tb.Close()
}

if isSafe, _ := netPlugin.Plugin.IsSafeToRemoveLock(name); isSafe {
log.Printf("[CNI] Removing lock file as process holding lock exited")
if errUninit := netPlugin.Plugin.UninitializeKeyValueStore(true); errUninit != nil {
log.Errorf("Failed to uninitialize key-value store of network plugin, err:%v.\n", errUninit)
}
}

return
}

Expand Down
15 changes: 0 additions & 15 deletions cni/plugin.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,28 +163,13 @@ func (plugin *Plugin) InitializeKeyValueStore(config *common.PluginConfig) error
removeLockFileAfterReboot(plugin)
}

// Acquire store lock.
if err := plugin.Store.Lock(true); err != nil {
log.Printf("[cni] Failed to lock store: %v.", err)
return err
}

config.Store = plugin.Store

return nil
}

// Uninitialize key-value store
func (plugin *Plugin) UninitializeKeyValueStore(force bool) error {
if plugin.Store != nil {
err := plugin.Store.Unlock(force)
if err != nil {
log.Printf("[cni] Failed to unlock store: %v.", err)
return err
}
}
plugin.Store = nil

return nil
}

Expand Down
9 changes: 8 additions & 1 deletion store/json.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,8 +221,15 @@ func (kvs *jsonFileStore) Lock(block bool) error {

defer lockFile.Close()

currentPid := os.Getpid()
log.Printf("Write pid %d to lockfile", currentPid)
// Write the process ID for easy identification.
if _, err = lockFile.WriteString(strconv.Itoa(os.Getpid())); err != nil {
if _, err = lockFile.WriteString(strconv.Itoa(currentPid)); err != nil {
// remove lockfile
log.Errorf("Write to lockfile failed:%+v", err)
if errRem := os.Remove(kvs.lockFileName); errRem != nil {
log.Errorf("removing lockfile failed:%+v", errRem)
}
return err
}

Expand Down