fix: do not continue processing records on permission errors in schema registry

[lucasbru]

Description

KSQL currently treats an exception due to missing permission on schema registry during deserialization as any other deserialization exception, and the default action for deserialization exception is log and continue.

The permission issue should stop KSQL from processing.

With this change:

• We scan the exception in the LogMetricAndContinueExceptionHandler for a RestClientException (can have multiple levels of wrapping) to get the error code and check if we have to deal with an authorization error.
• Return FAIL from the deserializationExceptionHandler for such exceptions.
• The exception will fall through to StreamsUncaughtExceptionHandler. There, we make sure that the exception is classified as a USER error so that on-call is not alerted. The uncaught exception handler will restart the thread and retry processing records.

The StreamThreads for that query will end up in a loop that will restart the StreamThread and retry processing the next record. So we effectively get that the query is blocked until the permission error is resolved.

Testing done

Tested the change in a local setup connected to ccloud schema registry.

Unit tests included.

Reviewer checklist

• Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
• Ensure relevant issues are linked (description should include text like "Fixes #")
• Do these changes have compatibility implications for rollback? If so, ensure that the ksql command version is bumped.

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[spena]

I left a couple of comments of using the Throwables class instead of using recursion ourselves. The logic is good, so +1 for this PR. But If you can use Throwables, that will make the code simpler I think.

[lucasbru]

@spena thanks for the review. I updated the code, seems like you'll need to reapprove it

[suhas-satish]

Approved based on Sergio's recent comment , I see throwable has been used. Thanks @lucasbru