[QE] Allow override transport for wathola sender with plugin system

[cardil]

Problem
Currently, wathola sender, a tool used in eventing upgrade tests, can send events only using HTTP POST request to a specified URL. That's okay for most of the tests, but in some cases it's lacking. Mostly for testing various Knative sources. In those tests we would like to send events directly to a given system like for ex.: Kafka's topic to be consumed by source under test.

Persona:
Developer

Exit Criteria
A set of Golang interfaces should be created, for sending events, and for transport configuration. The configuration should be added to allow override default HTTP Post transport with custom one provided.

After fixing this issue, it should be possible to use wathola tool to write tests for knative-extensions/eventing-kafka#67

Time Estimate (optional):
2d

[cardil]

/area test-and-release
/assign

[zhongduo]

You might want to check out the cloudevents sdk-go, I think it has some binding and transport supports.

[cardil]

@zhongduo right, there's gochan protocol that I can use to send an event to a different place, but that will also require some change in sender code to allow influencing that from outside packages.

[vaikas]

@slinkydeveloper do we think this might be functionality for the eventshub since it's using the cloudevents, etc.?

https://github.com/knative-sandbox/reconciler-test/tree/master/cmd/eventshub

[slinkydeveloper]

do we think this might be functionality for the eventshub since it's using the cloudevents, etc.?

Maybe? I'm not really sure what's the requirement here.

I do actually wonder why we don't use eventshub (which is meant to be the one stop for all utilities to send/receive/transform events) for upgrade tests

[vaikas]

@slinkydeveloper that's why I wanted to loop you in here to see if we can utilize eventshub for all the needs. I wasn't sure if maybe it was missing something. looks like we want to maybe have something like kafka sender, and for example in knative-sandbox/eventing-rabbitmq we have a sender that knows how to send native rabbitmq messages to exercise the source.

[cardil]

@slinkydeveloper I'm not aware of this eventshub thing. Can you provide some knowledge links, so I can read more what is the purpose of it?

On the other hand I'll try to provide some background here for you what upgrade tests are doing with their wathola tool.

The overall picture can be seen in https://github.com/knative/eventing/blob/master/test/upgrade/README.md#probe-test paragraph.

TL;DR - it has multiple distributed services, that continually (with constant interval) sends numbered events, receive them, and verify that all of them were propagated at least once. It also reports doubles. Recently, @zhongduo added capability to detect and report unavailability periods.

So, it is a tool written to assert continual, proper, operation of eventing system.

As of now this tool utilizes cloudevents HTTP sender. This issue and draft PR #4815 is to expand that capability to enable use cases @vaikas mention: kafka native sender, rabbitmq native sender, etc. Those would allow us to reliably tests various Knative sources.

[slinkydeveloper]

@cardil eventshub is able to perform the same tasks, with the difference that today it doesn't record the aggregation of events, but it record them one by one https://github.com/knative-sandbox/reconciler-test/tree/master/pkg/test_images/eventshub
eventshub "toolkit" includes the code to perform assertions on the received events. All the mentioned features actually belong to eventshub I believe. Would you be game in exploring a port of the wathola image there?

I would love to unify our work, It allows us to reduce the overall amount of test images, which would be great. We have a lot of test code/test images with consequent tech debt we're trying to get rid of, and, as far as I see from the opened issues about wathola, I think we're aligned in the goals we're going for here and I'd love if we could take a more integrated approach towards the eventshub.

[cardil]

@slinkydeveloper I agree that we should unify those two in single configurable tooling.

But, I don't think, I should make this quite large unifying effort while addressing this issue.

Maybe even more, judging by features alone, it looks to me like wathola already have more features useful for upgrade tests then eventshub. That's why I'm not entirely convinced, that we should move wathola features to enventshub, and not the other way around.

[slinkydeveloper]

But, I don't think, I should make this quite large unifying effort while addressing this issue.

What I'm saying is that we should address first the problem of having 2 test images with similar goals, and then move forward with addressing the remaining issues.

That's why I'm not entirely convinced, that we should move wathola features to enventshub, and not the other way around.

That's ok too for me, as long as we keep the "result image" in reconciler-test repo and it keeps the same features and the same assertion system (publish events via k8s events with EventInfo structure, all the various reply options, etc).

[cardil]

I have to do knative-extensions/eventing-kafka#67, so I really would like to do it here. And then make that refactoring unifying effort.

The other way around would make postponing knative-extensions/eventing-kafka#67 quite a lot, and that's not good.

[slinkydeveloper]

Can we have this change to wathola "contained" only in eventing-kafka as a "temporary solution" and then we put our hands on the refactoring/unification effort? Sooner we unify that code, better is for everybody and simpler is to perform such unification. If we diverge too much in features, the unification task will just become more complex

[cardil]

@slinkydeveloper I can agree to such plan.

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.