Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable NodeFeature API by default #1329

Merged
merged 1 commit into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/nfd-master/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ func initFlags(flagset *flag.FlagSet) (*master.Args, *master.ConfigOverrideArgs)
"Config file to use.")
flagset.StringVar(&args.Kubeconfig, "kubeconfig", "",
"Kubeconfig to use")
flagset.BoolVar(&args.EnableNodeFeatureApi, "enable-nodefeature-api", false,
flagset.BoolVar(&args.EnableNodeFeatureApi, "enable-nodefeature-api", true,
"Enable the NodeFeature CRD API for receiving node features. This will automatically disable the gRPC communication.")
flagset.BoolVar(&args.CrdController, "featurerules-controller", true,
"Enable NFD CRD API controller. DEPRECATED: use -crd-controller instead")
Expand Down
2 changes: 1 addition & 1 deletion cmd/nfd-worker/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ func initFlags(flagset *flag.FlagSet) (*worker.Args, *worker.ConfigOverrideArgs)
"Config file to use.")
flagset.StringVar(&args.KeyFile, "key-file", "",
"Private key matching -cert-file")
flagset.BoolVar(&args.EnableNodeFeatureApi, "enable-nodefeature-api", false,
flagset.BoolVar(&args.EnableNodeFeatureApi, "enable-nodefeature-api", true,
"Enable the NodeFeature CRD API for communicating with nfd-master. This will automatically disable the gRPC communication.")
flagset.StringVar(&args.Kubeconfig, "kubeconfig", "",
"Kubeconfig to use")
Expand Down
2 changes: 1 addition & 1 deletion deployment/helm/node-feature-discovery/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ nameOverride: ""
fullnameOverride: ""
namespaceOverride: ""

enableNodeFeatureApi: false
enableNodeFeatureApi: true

master:
config: ### <NFD-MASTER-CONF-START-DO-NOT-REMOVE>
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment/helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ We have introduced the following Chart parameters.
| `fullnameOverride` | string | | Override a default fully qualified app name |
| `tls.enable` | bool | false | Specifies whether to use TLS for communications between components |
| `tls.certManager` | bool | false | If enabled, requires [cert-manager](https://cert-manager.io/docs/) to be installed and will automatically create the required TLS certificates |
| `enableNodeFeatureApi` | bool | false | Enable the [NodeFeature](../usage/custom-resources.md#nodefeature) CRD API for communicating node features. This will automatically disable the gRPC communication.
| `enableNodeFeatureApi` | bool | true | Enable the [NodeFeature](../usage/custom-resources.md#nodefeature) CRD API for communicating node features. This will automatically disable the gRPC communication.
| `prometheus.enable` | bool | false | Specifies whether to expose metrics using prometheus operator |
| `prometheus.labels` | dict | {} | Specifies labels for use with the prometheus operator to control how it is selected |

Expand Down
4 changes: 2 additions & 2 deletions docs/get-started/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,8 @@ only created on nodes running nfd-master.

NFD takes use of some Kubernetes Custom Resources.

[NodeFeature](../usage/custom-resources.md#nodefeature)s (EXPERIMENTAL)
can be used for representing node features and requesting node labels to be
[NodeFeature](../usage/custom-resources.md#nodefeature)s
ArangoGutierrez marked this conversation as resolved.
Show resolved Hide resolved
is be used for representing node features and requesting node labels to be
generated.

NFD-Master uses [NodeFeatureRule](../usage/custom-resources.md#nodefeaturerule)s
Expand Down
9 changes: 5 additions & 4 deletions docs/reference/master-commandline-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,16 +153,17 @@ nfd-master -verify-node-name -ca-file=/opt/nfd/ca.crt \

### -enable-nodefeature-api

The `-enable-nodefeature-api` flag enables the
The `-enable-nodefeature-api` flag enables/disables the
[NodeFeature](../usage/custom-resources.md#nodefeature) CRD API for receiving
feature requests. This will also automatically disable the gRPC interface.
feature requests. This will also automatically disable/enable the gRPC
interface.

Default: false
Default: true

Example:

```bash
nfd-master -enable-nodefeature-api
nfd-master -enable-nodefeature-api=false
```

### -enable-leader-election
Expand Down
12 changes: 6 additions & 6 deletions docs/reference/worker-commandline-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,19 +197,19 @@ nfd-worker -label-sources=kernel,system,local

### -enable-nodefeature-api

The `-enable-nodefeature-api` flag enables the experimental
The `-enable-nodefeature-api` flag enables/disables the
[NodeFeature](../usage/custom-resources.md#nodefeature) CRD API
for communicating with nfd-master. This will also automatically disable the
gRPC communication to nfd-master. When enabled, nfd-worker will create per-node
for communicating with nfd-master. When enabled nfd-worker creates per-node
NodeFeature objects the contain all discovered node features and the set of
feature labels to be created.
feature labels to be created. Setting the flag to false will enable
gRPC communication to nfd-master.

Default: false
Default: true

Example:

```bash
nfd-worker -enable-nodefeature-api
nfd-worker -enable-nodefeature-api=false
```

### -metrics
Expand Down
10 changes: 4 additions & 6 deletions docs/usage/custom-resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,11 @@ NFD uses some Kubernetes [custom resources][custom-resources].

## NodeFeature

**EXPERIMENTAL**
ArangoGutierrez marked this conversation as resolved.
Show resolved Hide resolved
NodeFeature is an NFD-specific custom resource for communicating node
features and node labeling requests. Support for NodeFeature objects is
disabled by default. If enabled, nfd-master watches for NodeFeature objects,
labels nodes as specified and uses the listed features as input when evaluating
[NodeFeatureRule](#nodefeaturerule)s. NodeFeature objects can be used for
implementing 3rd party extensions (see
features and node labeling requests. The nfd-master pod watches for NodeFeature
objects, labels nodes as specified and uses the listed features as input when
evaluating [NodeFeatureRule](#nodefeaturerule)s. NodeFeature objects can be
used for implementing 3rd party extensions (see
[customization guide](customization-guide.md#nodefeature-custom-resource) for more
details).

Expand Down
10 changes: 5 additions & 5 deletions docs/usage/customization-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ sort: 8
NFD provides multiple extension points for vendor and application specific
labeling:

- [`NodeFeature`](#nodefeature-custom-resource) (EXPERIMENTAL) objects can be
- [`NodeFeature`](#nodefeature-custom-resource) objects can be
used to communicate "raw" node features and node labeling requests to
nfd-master.
- [`NodeFeatureRule`](#nodefeaturerule-custom-resource) objects provide a way to
Expand All @@ -32,7 +32,6 @@ labeling:

## NodeFeature custom resource

**EXPERIMENTAL**
ArangoGutierrez marked this conversation as resolved.
Show resolved Hide resolved
NodeFeature objects provide a way for 3rd party extensions to advertise custom
features, both as "raw" features that serve as input to
[NodeFeatureRule](#nodefeaturerule-custom-resource) objects and as feature
Expand All @@ -41,9 +40,10 @@ labels directly.
Note that RBAC rules must be created for each extension for them to be able to
create and manipulate NodeFeature objects in their namespace.

Support for NodeFeature CRD API is enabled with the `-enable-nodefeature-api`
command line flag. This flag must be specified for both nfd-master and
nfd-worker as it will disable the gRPC communication between them.
The NodeFeature CRD API can be disabled with the
`-enable-nodefeature-api=false` command line flag. This flag must be specified
for both nfd-master and nfd-worker as it will enable the gRPC communication
between them.

### A NodeFeature example

Expand Down
40 changes: 14 additions & 26 deletions docs/usage/nfd-master.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,40 +15,26 @@ extended resources based on requests from nfd-workers and 3rd party extensions.

## NodeFeature controller

**EXPERIMENTAL**
ArangoGutierrez marked this conversation as resolved.
Show resolved Hide resolved
Controller for [NodeFeature](custom-resources.md#nodefeature-custom-resource)
objects can be enabled with the
[`-enable-nodefeature-api`](../reference/master-commandline-reference.md#-enable-nodefeature-api)
command line flag. When enabled, features from NodeFeature objects are used as
The NodeFeature Controller uses NodeFeature objects as
the input for the [NodeFeatureRule](custom-resources.md#nodefeaturerule)
processing pipeline. In addition, any labels listed in the NodeFeature object
are created on the node (note the allowed
[label namespaces](customization-guide.md#node-labels) are controlled).

> **NOTE:** NodeFeature API must also be enabled in nfd-worker with
> its [`-enable-nodefeature-api`](../reference/worker-commandline-reference.md#-enable-nodefeature-api)
> flag.

When `-enable-nodefeature-api` option is enabled and NFD-Master is intended to run
with more than one replica, it is advised to use `-enable-leader-election` flag.
This flag turns on leader election for NFD-Master and let only one replica
to act on changes in NodeFeature and NodeFeatureRule objects.

## NodeFeatureRule controller

NFD-Master acts as the controller for
[NodeFeatureRule](custom-resources.md#nodefeaturerule) objects.
It applies the rules specified in NodeFeatureRule objects on raw feature data
and creates node labels accordingly. The feature data used as the input can be
received from nfd-worker instances through the gRPC interface or from
[NodeFeature](custom-resources.md#nodefeature-custom-resource) objects. The latter
requires that the [NodeFeaure controller](#nodefeature-controller) has been
enabled.

> **NOTE:** when gRPC is used for communicating the features (the default
> mechanism), (re-)labelling only happens when a request is received from
> nfd-worker. That is, in practice rules are evaluated and labels for each node
> are created on intervals specified by the
and creates node labels accordingly. The feature data used as the input is
received from nfd-worker instances through
[NodeFeature](custom-resources.md#nodefeature-custom-resource) objects.

> **NOTE:** when gRPC is used for communicating the features (by setting the
> flag `-enable-nodefeature-api=false` on both nfd-master and nfd-worker, or
> via Helm values.enableNodeFeatureApi=false),(re-)labelling only happens
> when a request is received from nfd-worker. That is, in practice rules are
> evaluated and labels for each node are created on intervals specified by the
> [`core.sleepInterval`](../reference/worker-configuration-reference.md#coresleepinterval)
> configuration option of nfd-worker instances. This means that modification or
> creation of NodeFeatureRule objects does not instantly cause the node
Expand Down Expand Up @@ -103,8 +89,10 @@ affinity to prevent masters from running on the same node.
However note that inter-pod affinity is costly and is not recommended
in bigger clusters.

> **NOTE:** If the [NodeFeature controller](#nodefeature-controller) is enabled
> the replica count should be 1.
> **Note:** When NFD-Master is intended to run with more than one replica,
> it is advised to use `-enable-leader-election` flag. This flag turns on
> leader election for NFD-Master and let only one replica to act on changes
> in NodeFeature and NodeFeatureRule objects.

If you have RBAC authorization enabled (as is the default e.g. with clusters
initialized with kubeadm) you need to configure the appropriate ClusterRoles,
Expand Down
4 changes: 2 additions & 2 deletions test/e2e/node_feature_discovery_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,8 @@ var _ = SIGDescribe("NFD master and worker", func() {

nfdTestSuite := func(useNodeFeatureApi bool) {
createPodSpecOpts := func(opts ...testpod.SpecOption) []testpod.SpecOption {
if useNodeFeatureApi {
return append(opts, testpod.SpecWithContainerExtraArgs("-enable-nodefeature-api"))
if !useNodeFeatureApi {
return append(opts, testpod.SpecWithContainerExtraArgs("-enable-nodefeature-api=false"))
}
return opts
}
Expand Down