Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace option.Config.{Get,Set,Append}Devices by table lookups #30578

Merged
merged 14 commits into from
Apr 15, 2024

Conversation

bimmlerd
Copy link
Member

@bimmlerd bimmlerd commented Feb 1, 2024

This replaces usages of the Get/SetDevices APIs by lookups to the device table, and includes the necessary plumbing of that everywhere.

Scary stuff includes the wireguard interface thing in daemon/loader, not sure my transformation is functionally equivalent. CI will hopefully tell.

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Feb 1, 2024
@bimmlerd bimmlerd added kind/cleanup This includes no functional changes. release-note/misc This PR makes changes that have no direct user impact. dont-merge/blocked Another PR must be merged before this one. sig/agent Cilium agent related. area/modularization labels Feb 1, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Feb 1, 2024
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch 6 times, most recently from 2fd8348 to 782348a Compare February 2, 2024 08:54
@bimmlerd
Copy link
Member Author

bimmlerd commented Feb 2, 2024

/test

Copy link

github-actions bot commented Mar 4, 2024

This pull request has been automatically marked as stale because it
has not had recent activity. It will be closed if no further activity
occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Mar 4, 2024
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from 782348a to 81564f7 Compare March 18, 2024 11:27
@bimmlerd bimmlerd removed the dont-merge/blocked Another PR must be merged before this one. label Mar 18, 2024
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from 81564f7 to 53e083e Compare March 18, 2024 11:50
@bimmlerd
Copy link
Member Author

/test

@github-actions github-actions bot removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Mar 19, 2024
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from 53e083e to 2f9afc0 Compare March 25, 2024 10:02
@bimmlerd
Copy link
Member Author

/test

@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from 2f9afc0 to b568561 Compare March 25, 2024 14:53
@bimmlerd
Copy link
Member Author

/ci-e2e

@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from b568561 to 5384d27 Compare March 26, 2024 13:25
@bimmlerd
Copy link
Member Author

/ci-e2e

pkg/datapath/loader/cache_test.go Outdated Show resolved Hide resolved
pkg/datapath/loader/cache_test.go Outdated Show resolved Hide resolved
pkg/datapath/loader/template_test.go Outdated Show resolved Hide resolved
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from 5384d27 to 1d80593 Compare April 2, 2024 09:13
Copy link
Contributor

@tommyp1ckles tommyp1ckles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Endpoint changes look good.

daemon/cmd/config.go Show resolved Hide resolved
pkg/datapath/loader/template_test.go Outdated Show resolved Hide resolved
@bimmlerd bimmlerd force-pushed the pr/bimmlerd/remove-get-set-devices branch from a799c4f to 0e2eb4b Compare April 9, 2024 15:19
@bimmlerd
Copy link
Member Author

bimmlerd commented Apr 9, 2024

/test

@joamaki joamaki force-pushed the pr/bimmlerd/remove-get-set-devices branch from 0e2eb4b to 919b2e6 Compare April 12, 2024 10:04
@joamaki
Copy link
Contributor

joamaki commented Apr 12, 2024

/test

@joamaki joamaki enabled auto-merge April 12, 2024 10:12
@joamaki joamaki force-pushed the pr/bimmlerd/remove-get-set-devices branch 2 times, most recently from 6dbcf6f to 2455787 Compare April 12, 2024 10:45
@joamaki
Copy link
Contributor

joamaki commented Apr 12, 2024

/test

bimmlerd and others added 14 commits April 15, 2024 10:56
Devices (i.e. network interfaces) can change at runtime. The current
device API, however, suggest that these devices never change. Indeed, a
large portion of the agent is written with that assumption (though
workarounds have been added where changes to the devices broke things).

As part of an ongoing process to migrate the agent towards a future
where a changing set of devices is the norm, deprecate the existing,
snapshot-oriented APIs and replace it with table lookups.

In subsequent commits, usage of this API is reduced, and replaced with
usage of an API which allows for being notified when the set of devices
changes.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
This is handy for future commits in which we increase the number of
dependencies we'd have to manually construct when creating a linux
NodeHandler.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
* Refactor out the duplicated code to assert against NeighList
* Use proper fake nodemap in TestNodeUpdateIDs

Signed-off-by: Jussi Maki <jussi@isovalent.com>
Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
While the tests claim they need the full datapath, that is no longer
true. Instead, provide a fake datapath (to avoid having to provide all
the dependencies).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
GetDevices returns a snapshot, but devices can change at runtime. The
new, table-based API reflects this fact. This patch, however, does not
change the semantics - it merely replaces the usage of GetDevices with a
single table lookup, to allow us to react to changes in the future (and
remove the GetDevices API).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
GetDevices returns a snapshot, but devices can change at runtime. The
new, table-based API reflects this fact. This patch, however, does not
change the semantics - it merely replaces the usage of GetDevices with a
single table lookup, to allow us to react to changes in the future (and
remove the GetDevices API).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
GetDevices returns a snapshot, but devices can change at runtime. The
new, table-based API reflects this fact. This patch, however, does not
change the semantics - it merely replaces the usage of GetDevices with a
single table lookup, to allow us to react to changes in the future (and
remove the GetDevices API).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
There are no usages, hence it is safe to remove.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
GetDevices returns a snapshot, but devices can change at runtime. The
new, table-based API reflects this fact. This patch, however, does not
change the semantics - it merely replaces the usage of GetDevices with a
single table lookup, to allow us to react to changes in the future (and
remove the GetDevices API).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
GetDevices returns a snapshot, but devices can change at runtime. The
new, table-based API reflects this fact. This patch, however, does not
change the semantics - it merely replaces the usage of GetDevices with a
single table lookup, to allow us to react to changes in the future (and
remove the GetDevices API).

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
As explained in 0670fd2, the WireGuard
device must be considered for loading bpf_host in certain conditions.

With the removal of the Get/Set/AppendDevices APIs, the global addition
of the WG interface to devices is no longer possible, nor was it
ever really desirable.

Since we don't want to make WG a "selected" device, we move the
inclusion of this interface to the two places necessary: when writing
static EP data in the config writer, as well as the loading of bpf_host.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
The agent needs to choose between loading bpf_network or bpf_host onto
physical devices. In the long-term, we want to get rid of bpf_network.
In the short-term, however, it's still around, so we need to keep the
existing semantics intact.

Previous code chose by checking the devices configuration (i.e.
--devices) as well as the runtime-derived list of
devices, to understand whether we were auto-detecting devices. If so,
bpf_network would not be loaded in reinitializeIPSec, as bpf_host would
be loaded.

With the change towards always auto detecting devices, these checks need
to be switched for AreDevicesRequired in the short-term, a dedicated
flag in the long term to allow clean fall-back to encryptInterface and
the removal of bpf_network in the longer term.

Co-authored-by: Jussi Maki <jussi@isovalent.com>
Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
Since the devices controller is really the owner of this flag, move the
configuration into the cell proper, to avoid other things being able to
depend on it.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
The snapshot based APIs are impossible to use correctly, as they do not
reflect the fact that the set of devices can change at runtime. Uses of
this API have been replaced with lookups into the Table[Device] in the
past few commits, hence removal is now easy.

Signed-off-by: David Bimmler <david.bimmler@isovalent.com>
@joamaki joamaki force-pushed the pr/bimmlerd/remove-get-set-devices branch from 2455787 to 6e93393 Compare April 15, 2024 08:57
@dylandreimerink
Copy link
Member

/test

@joamaki joamaki added this pull request to the merge queue Apr 15, 2024
Merged via the queue into cilium:main with commit 5d1521b Apr 15, 2024
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/modularization kind/cleanup This includes no functional changes. release-note/misc This PR makes changes that have no direct user impact. sig/agent Cilium agent related.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants