Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1150 Console plugin columns & filters as config #417

Merged
merged 9 commits into from
Nov 7, 2023

Conversation

jpinsonneau
Copy link
Contributor

@jpinsonneau jpinsonneau commented Oct 20, 2023

Description

This PR allow sending definitions from config for:

  • columns - default to EndTime + Src / Dst addresses
    • set id, name, group and helpers from texts
    • size from number
    • default selection from boolean
    • map to an existing filter using filter id
    • map value and sort functions according to field or calculated value
  • filters - optionnal (filter bar is hidden if not set)
    • set id, name and hints from texts
    • set component type from text
    • map options, encoder and validation from id keywords.

It also move server and loki configs to the loaded configmap for consistency.

image
image
image

/!\ FYI some behaviors are still hardcoded by column id.
I suggest to slowly migrate to something more flexible according to our needs.

Dependencies

netobserv/network-observability-operator#477

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@codecov
Copy link

codecov bot commented Oct 26, 2023

Codecov Report

Attention: 159 lines in your changes are missing coverage. Please review.

Comparison is base (6d73e0a) 58.09% compared to head (d796f6d) 57.57%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #417      +/-   ##
==========================================
- Coverage   58.09%   57.57%   -0.52%     
==========================================
  Files         169      170       +1     
  Lines        7985     8048      +63     
  Branches      990      975      -15     
==========================================
- Hits         4639     4634       -5     
- Misses       3067     3139      +72     
+ Partials      279      275       -4     
Flag Coverage Δ
uitests 58.78% <71.28%> (+0.16%) ⬆️
unittests 54.27% <31.53%> (-2.30%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
pkg/server/routes.go 89.65% <100.00%> (ø)
pkg/server/server.go 70.00% <100.00%> (ø)
web/src/api/ipfix.ts 100.00% <100.00%> (ø)
web/src/components/__tests-data__/columns.ts 100.00% <100.00%> (ø)
web/src/components/__tests-data__/config.ts 100.00% <100.00%> (ø)
web/src/components/__tests-data__/filters.ts 100.00% <100.00%> (ø)
web/src/components/__tests-data__/flows.ts 100.00% <100.00%> (ø)
web/src/components/filters/compare-filter.tsx 87.80% <100.00%> (-0.30%) ⬇️
web/src/components/filters/filters-dropdown.tsx 95.45% <100.00%> (ø)
web/src/components/filters/filters-helper.ts 100.00% <100.00%> (ø)
... and 26 more

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

enable: true
portNames:
"3100": loki
columns:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all the config below: do you remember if you changed some content or if it's only moving around these strings? (Just to help me reviewing, because I can't see the diff)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed a bit the fields:

  • fieldName -> field
  • isSelected -> default
    and introduced calculated field for more complex value binding.

Appart from that it's the same as before. I'm keeping the yaml updated according to parallel PRs in both example and operator PR. See:
netobserv/network-observability-operator#477 (comment)

const previousObj = usePrevious(obj);
const previous = usePrevious({ obj, state: initState.current });

React.useEffect(() => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same logic here is done in netflow-traffic component .. would it make sense to move that in the parent, and pass it down to its children? (unless it's not as simple as that)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The init state of the tab is not the same as the init state of netflow traffic here.

I've replicated the getConfig part that could be shared in a common parent but it will need a bit of refactoring and to find a way to discriminate tabs from page.
Currently we are pointing different components using "$codeRef": "netflowParent.default" / "$codeRef": "netflowTab.default"

If you want we can try that in a followup

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no no that's fine, I thought it was really the same thing

Destination,
None
}
export type FilterCategory = 'source' | 'destination';

export type TargetedFilterId =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a user configures a filter with an id that isn't in that list? (e.g. imagine there's a new field we forgot to add here) - Is it still going to work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing. The columns are still working fine since it's javascript behind the scene.
However, note that some behaviors are still hardcoded by column id as written in the description above.

Comment on lines +252 to +277
if (d.id.includes('namespace')) {
getOptions = cap10(getNamespaceOptions);
validate = k8sNameValidation;
} else if (d.id.includes('name')) {
validate = k8sNameValidation;
} else if (d.id.includes('kind')) {
getOptions = cap10(getKindOptions);
validate = rejectEmptyValue;
encoder = kindFiltersEncoder(`${isSrc ? 'Src' : 'Dst'}K8S_Type`, `${isSrc ? 'Src' : 'Dst'}K8S_OwnerType`);
} else if (d.id.includes('resource')) {
getOptions = cap10(getResourceOptions);
validate = k8sResourceValidation;
checkCompletion = k8sResourceCompletion;
encoder = k8sResourceFiltersEncoder(
`${isSrc ? 'Src' : 'Dst'}K8S_Type`,
`${isSrc ? 'Src' : 'Dst'}K8S_OwnerType`,
`${isSrc ? 'Src' : 'Dst'}K8S_Namespace`,
`${isSrc ? 'Src' : 'Dst'}K8S_Name`,
`${isSrc ? 'Src' : 'Dst'}K8S_OwnerName`
);
} else if (d.id.includes('address')) {
validate = addressValidation;
} else if (d.id.includes('port')) {
getOptions = cap10(getPortOptions);
validate = portValidation;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if, rather than that, it would be better to name the bunch of "validators" defined above (ie. the validation functions) and refer to these names in the config ? That would be more in line with the desired config-driven approach ?

Copy link
Member

@jotak jotak Nov 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, I admit that things like that make it slighlty more complicated:

      encoder = k8sResourceFiltersEncoder(
        `${isSrc ? 'Src' : 'Dst'}K8S_Type`,
        `${isSrc ? 'Src' : 'Dst'}K8S_OwnerType`,
        `${isSrc ? 'Src' : 'Dst'}K8S_Namespace`,
        `${isSrc ? 'Src' : 'Dst'}K8S_Name`,
        `${isSrc ? 'Src' : 'Dst'}K8S_OwnerName`
      );

I don't know ... up to you :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. As same as columns, we need to find a tradeoff between config and hardcoded switches.
For now I choosed a simple approach for simplicity to introduce the config without breaking changes.

Ideally, I would like to have a way to write predefined javascript functions as same as calculated field from columns with more capabilities.

jotak
jotak previously approved these changes Nov 6, 2023
Copy link
Member

@jotak jotak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jotak
Copy link
Member

jotak commented Nov 7, 2023

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Nov 7, 2023
@openshift-ci openshift-ci bot added the lgtm label Nov 7, 2023
@jotak jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Nov 7, 2023
Copy link

github-actions bot commented Nov 7, 2023

New image:
quay.io/netobserv/network-observability-console-plugin:11f1f1b

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=11f1f1b make set-plugin-image

Copy link

openshift-ci bot commented Nov 7, 2023

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jpinsonneau jpinsonneau added the no-qe This PR doesn't necessitate QE approval label Nov 7, 2023
@openshift-merge-bot openshift-merge-bot bot merged commit be4ea45 into netobserv:main Nov 7, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm no-qe This PR doesn't necessitate QE approval ok-to-test To set manually when a PR is safe to test. Triggers image build on PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants