Upgrade tests for KafkaSource

[aliok]

We need to implement upgrade tests for KafkaSource and KafkaBinding so that in the future we can change the storage version to v1beta1 from v1alpha1.

Best idea is to keep eventing at latest release and only changing the eventing-kafka version. We can in the future run the tests again with eventing head too.

Part of umbrella issue, #65

[cardil]

I think this should be modeled similar to eventing and serving upgrade tests. That means upgrading to the latest version from stable and going back.

If we implement migration of API at any point it should do that in that migration period and tests should survive that.

My approach would be to adopt a new upgrade test framework for compose-ability and write some standard pre- post- assertions and continual tests.

To be able to write continual tests, I'm thinking to extend Wathola testing utility from eventing, so it will be possible to replace its sender transport from HTTP request to a Kafka client.

That would enable us to send events directly to Kafka topic, so we can configure KafkaSource or/and KafkaBinding and feed cloud events to broker (in future this broker could be backed by KafkaChannel #72), and then to the receiver for validation.

/assign

[matzew]

@cardil I agree that using the schema/model from serving/eventing is right

What do you mean with:

adopt a new upgrade

Also, extending the utils from eventing to have usage on downstream seems good 👍

[cardil]

I could also skip extending testing tools and use KafkaSink. But, KafkaSink isn't in this repo, right?

[cardil]

/area test-and-release

[knative-prow-robot]

@cardil: The label(s) area/test-and-release cannot be applied, because the repository doesn't have them

In response to this:

/area test-and-release

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cardil]

/reopen

[knative-prow-robot]

@cardil: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.