Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] feat: add constructor for full event lineage graph #7921

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Cali0707
Copy link
Member

Fixes #7694

Proposed Changes

  • Get resources from cluster and construct a graph

Signed-off-by: Calum Murray <cmurray@redhat.com>
@knative-prow knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 13, 2024
@Cali0707
Copy link
Member Author

/cc @pierDipi

@knative-prow knative-prow bot requested a review from pierDipi May 13, 2024 18:41
Copy link

knative-prow bot commented May 13, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot requested review from aslom and matzew May 13, 2024 18:41
@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 13, 2024
Comment on lines +59 to +121
apiServerSources, err := eventingClient.SourcesV1().ApiServerSources("").List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}

for _, apiServerSource := range apiServerSources.Items {
if filterFunc(apiServerSource) {
g.AddSource(duckv1.Source{
ObjectMeta: apiServerSource.ObjectMeta,
TypeMeta: apiServerSource.TypeMeta,
Spec: apiServerSource.Spec.SourceSpec,
Status: apiServerSource.Status.SourceStatus,
})
}
}

containerSources, err := eventingClient.SourcesV1().ContainerSources("").List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}

for _, containerSource := range containerSources.Items {
if filterFunc(containerSource) {
g.AddSource(duckv1.Source{
ObjectMeta: containerSource.ObjectMeta,
TypeMeta: containerSource.TypeMeta,
Spec: containerSource.Spec.SourceSpec,
Status: containerSource.Status.SourceStatus,
})
}
}

pingSources, err := eventingClient.SourcesV1().PingSources("").List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}

for _, pingSource := range pingSources.Items {
if filterFunc(pingSource) {
g.AddSource(duckv1.Source{
ObjectMeta: pingSource.ObjectMeta,
TypeMeta: pingSource.TypeMeta,
Spec: pingSource.Spec.SourceSpec,
Status: pingSource.Status.SourceStatus,
})
}
}

sinkBindings, err := eventingClient.SourcesV1().SinkBindings("").List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}

for _, sinkBinding := range sinkBindings.Items {
if filterFunc(sinkBinding) {
g.AddSource(duckv1.Source{
ObjectMeta: sinkBinding.ObjectMeta,
TypeMeta: sinkBinding.TypeMeta,
Spec: sinkBinding.Spec.SourceSpec,
Status: sinkBinding.Status.SourceStatus,
})
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pierDipi do you have any ideas how we could get the sources (& in the duckv1.Source struct) more easily? This process is rather manual, and would not detect any sources not in eventing core :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is annotation/label on the CRDs that implement the source ducktype

duck.knative.dev/source: "true"

from the CRD list (with label selector) using dynamic client you can get the actual resource list

@Cali0707
Copy link
Member Author

This is WIP because I want feedback on the approach before adding unit tests

Signed-off-by: Calum Murray <cmurray@redhat.com>
Copy link

codecov bot commented May 13, 2024

Codecov Report

Attention: Patch coverage is 2.29885% with 85 lines in your changes are missing coverage. Please review.

Project coverage is 69.08%. Comparing base (7e1c082) to head (582ca21).
Report is 42 commits behind head on main.

Files Patch % Lines
pkg/graph/constructor.go 2.29% 85 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7921      +/-   ##
==========================================
- Coverage   69.22%   69.08%   -0.14%     
==========================================
  Files         339      344       +5     
  Lines       19494    16069    -3425     
==========================================
- Hits        13494    11101    -2393     
+ Misses       5337     4286    -1051     
- Partials      663      682      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines +37 to +40
brokers, err := eventingClient.EventingV1().Brokers("").List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for authorization errors should we try to build the rest of the graph before returning?

duckv1 "knative.dev/pkg/apis/duck/v1"
)

func ConstructGraph(ctx context.Context, filterFunc func(obj interface{}) bool) (*Graph, error) {
eventingClient := eventingclient.Get(ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd get this as an input instead of assuming injection context

Copy link
Member

@pierDipi pierDipi May 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the future proof way would be to have a struct as input so that we can evolve this function without having compilation errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Event Lineage: Create Graph from Cluster Resources
2 participants