[Conformance] Broker spec.config is not immutable as described in the spec

[salaboy]

Describe the bug
You can patch a broker spec.config field and you shouldn't be able to do that as it should be immutable as described here:
https://github.com/knative/specs/blob/main/specs/eventing/control-plane.md#broker-lifecycle

Expected behavior
You should get an error when you try to patch the spec.config of a broker.

To Reproduce

1. Create a Broker (my-broker)
2. Run: kubectl patch broker my-broker --type merge -p '{"spec":{"config":{"apiVersion":"v1"}}}'

Knative release version

v0.24.1

Additional context
The reference implementation should conform to the spec.