Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1517: remove IPFIX agent mode #579

Merged
merged 2 commits into from Mar 11, 2024
Merged

Conversation

jotak
Copy link
Member

@jotak jotak commented Feb 21, 2024

Description

  • Remove controller code for OVS configmap / CNO configuration
  • Remove related tests (and move console plugin related tests to its dedicated file)
  • Remove FLP ingester reconciler (was only used with ipfix)
  • Note that the IPFIX api is not removed to not introduce breaking changes in existing API, however it is now documented that using IPFIX would have no effect

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Feb 21, 2024

@jotak: This pull request references NETOBSERV-1517 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

  • Remove controller code for OVS configmap / CNO configuration
  • Remove related tests (and move console plugin related tests to its dedicated file)
  • Remove FLP ingester reconciler (was only used with ipfix)
  • Note that the IPFIX api is not removed to not introduce breaking changes in existing API, however it is now documented that using IPFIX would have no effect

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

openshift-ci bot commented Feb 21, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link

codecov bot commented Feb 21, 2024

Codecov Report

Attention: Patch coverage is 62.26415% with 20 lines in your changes are missing coverage. Please review.

Project coverage is 67.42%. Comparing base (3311834) to head (391bfd6).

Files Patch % Lines
controllers/consoleplugin/consoleplugin_objects.go 38.88% 7 Missing and 4 partials ⚠️
controllers/flp/flp_common_objects.go 52.94% 7 Missing and 1 partial ⚠️
controllers/ebpf/agent_controller.go 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #579      +/-   ##
==========================================
+ Coverage   67.16%   67.42%   +0.25%     
==========================================
  Files          71       65       -6     
  Lines        8417     7996     -421     
==========================================
- Hits         5653     5391     -262     
+ Misses       2411     2276     -135     
+ Partials      353      329      -24     
Flag Coverage Δ
unittests 67.42% <62.26%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 4, 2024

@jotak: This pull request references NETOBSERV-1517 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

  • Remove controller code for OVS configmap / CNO configuration
  • Remove related tests (and move console plugin related tests to its dedicated file)
  • Remove FLP ingester reconciler (was only used with ipfix)
  • Note that the IPFIX api is not removed to not introduce breaking changes in existing API, however it is now documented that using IPFIX would have no effect

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 6, 2024

@jotak: This pull request references NETOBSERV-1517 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

  • Remove controller code for OVS configmap / CNO configuration
  • Remove related tests (and move console plugin related tests to its dedicated file)
  • Remove FLP ingester reconciler (was only used with ipfix)
  • Note that the IPFIX api is not removed to not introduce breaking changes in existing API, however it is now documented that using IPFIX would have no effect

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

- Remove controller code for OVS configmap / CNO configuration
- Remove related tests (and move console plugin related tests to its
  dedicated file)
- Remove FLP ingester reconciler (was only used with ipfix)
- Note that the IPFIX api is not removed to not introduce breaking
  changes in existing API, however it is now documented that using IPFIX
would have no effect
Copy link
Contributor

@jpinsonneau jpinsonneau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the storage version has recently been changed to v1beta2 can't we get rid of the old IPFIX section + type field in this version ?

I don't think any release has been done in between.
That will help on #569 (comment)

return r.status.Error("ReconcileOVNKFailed", err)
}
}

// eBPF agent
ebpfAgentController := ebpf.NewAgentController(reconcilersInfo.NewInstance(r.mgr.Config.EBPFAgentImage, r.status))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not adding a test on agent type here and crash if IPFIX ?

That would be more relevent in case someone still uses this type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hesitated between the 2 approaches.. the thing I like with switching automatically to eBPF is that we completely remove any legacy IPFIX stuff from the code (except in the API) even that simple bool check & failure, so the state of the art is now: you have no choice it's eBPF. I have the feeling this is more user friendly than triggering an error & avoids risks of misconfiguration.

// but they would require manual configuration).
// `type` [deprecated (*)] selects the flows tracing agent. The only possible value is `EBPF` (default), to use NetObserv eBPF agent.<br>
// Previously, using an IPFIX collector was allowed, but was deprecated and it is now removed.<br>
// Setting `IPFIX` is ignored and still use the eBPF Agent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Setting `IPFIX` is ignored and still use the eBPF Agent.

// but they would require manual configuration).
// `type` [deprecated (*)] selects the flows tracing agent. The only possible value is `eBPF` (default), to use NetObserv eBPF agent.<br>
// Previously, using an IPFIX collector was allowed, but was deprecated and it is now removed.<br>
// Setting `IPFIX` is ignored and still use the eBPF Agent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Setting `IPFIX` is ignored and still use the eBPF Agent.

config/descriptions/upstream.md Outdated Show resolved Hide resolved
controllers/flp/flp_ingest_reconciler.go Show resolved Hide resolved
Copy link
Contributor

@OlivierCazade OlivierCazade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM, but I think we can simplify it much more now that we don't have IPFIX support. We now have the monolith and the transformer, the monolith is used without kafka and the transformer with kafka.

I think we can merge both and remove the flp_controller wrapper around them.

I am fine doing this later if you prefer.

@OlivierCazade OlivierCazade self-requested a review March 6, 2024 14:24
@jotak
Copy link
Member Author

jotak commented Mar 6, 2024

@jpinsonneau

Since the #577 can't we get rid of the old IPFIX section + type field in this version ?

That would be a breaking change which is forbidden in a version that has already been released. We need to create a new API version in order to remove fields (we should have done that before release v1beta2 the first time).

Co-authored-by: Olivier Cazade <olivier.cazade@gmail.com>
@jotak
Copy link
Member Author

jotak commented Mar 6, 2024

@OlivierCazade I'm actually removing the monolith reconciler in the other PR with unified agent+FLP (not pushed yet). So yes it's going to be even more simplified but via that other PR :)

@openshift-ci openshift-ci bot added the lgtm label Mar 6, 2024
@Amoghrd
Copy link
Contributor

Amoghrd commented Mar 7, 2024

/ok-to-test

@openshift-ci openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Mar 7, 2024
Copy link

github-actions bot commented Mar 7, 2024

New images:

  • quay.io/netobserv/network-observability-operator:bf47945
  • quay.io/netobserv/network-observability-operator-bundle:v0.0.0-bf47945
  • quay.io/netobserv/network-observability-operator-catalog:v0.0.0-bf47945

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:bf47945 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-bf47945

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-bf47945
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

Copy link

openshift-ci bot commented Mar 7, 2024

@jotak: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-operator 391bfd6 link false /test e2e-operator

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Amoghrd
Copy link
Contributor

Amoghrd commented Mar 8, 2024

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved QE has approved this pull request label Mar 8, 2024
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 8, 2024

@jotak: This pull request references NETOBSERV-1517 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

  • Remove controller code for OVS configmap / CNO configuration
  • Remove related tests (and move console plugin related tests to its dedicated file)
  • Remove FLP ingester reconciler (was only used with ipfix)
  • Note that the IPFIX api is not removed to not introduce breaking changes in existing API, however it is now documented that using IPFIX would have no effect

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jotak
Copy link
Member Author

jotak commented Mar 11, 2024

thanks @Amoghrd !
/approve

Copy link

openshift-ci bot commented Mar 11, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jotak jotak merged commit 323e3f8 into netobserv:main Mar 11, 2024
11 of 14 checks passed
@jotak jotak deleted the rm-ipfix branch March 11, 2024 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved jira/valid-reference lgtm ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. qe-approved QE has approved this pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants