KafkaConfigurationError: Unrecognized configs: {'kafka_client'}

[mortenlj]

I am trying to package karapace in Docker for use in the NAIS platform. I have used the work being done for #136 , because I need to base my image on our own baseimages.

I have managed to create an image with all dependencies as far as I can tell, but when I start the container, it fails with a KafkaConfigurationError. I'm not sure if this is a failure in my packaging, or an actual error in the current master branch?

The log:

Karapace            	MainThread	INFO    	Karapace initialized
KarapaceRest        	MainThread	ERROR   	Unable to start admin client, retrying
Traceback (most recent call last):
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/__init__.py", line 302, in init_admin_client
    self.admin_client = KafkaRestAdminClient(
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/admin.py", line 15, in __init__
    super().__init__(**configs)
  File "/app/.local/lib/python3.9/site-packages/kafka/admin/client.py", line 195, in __init__
    raise KafkaConfigurationError("Unrecognized configs: {}".format(extra_configs))
kafka.errors.KafkaConfigurationError: KafkaConfigurationError: Unrecognized configs: {'kafka_client'}
^CTraceback (most recent call last):
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/__init__.py", line 302, in init_admin_client
    self.admin_client = KafkaRestAdminClient(
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/admin.py", line 15, in __init__
    super().__init__(**configs)
  File "/app/.local/lib/python3.9/site-packages/kafka/admin/client.py", line 195, in __init__
    raise KafkaConfigurationError("Unrecognized configs: {}".format(extra_configs))
kafka.errors.KafkaConfigurationError: KafkaConfigurationError: Unrecognized configs: {'kafka_client'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.local/bin/karapace", line 8, in <module>
    sys.exit(main())
  File "/app/.local/lib/python3.9/site-packages/karapace/karapace_all.py", line 55, in main
    kc = KafkaRest(config_file_path=config_file_path, config=config)
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/__init__.py", line 43, in __init__
    self._init(config=config)
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/__init__.py", line 57, in _init
    self.init_admin_client()
  File "/app/.local/lib/python3.9/site-packages/karapace/kafka_rest_apis/__init__.py", line 316, in init_admin_client
    time.sleep(1)
KeyboardInterrupt
asyncio             	MainThread	ERROR   	Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f7875bb3d90>
asyncio             	MainThread	ERROR   	Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f7875983ac0>

I'm running with this configuration:

{
  "bootstrap_uri": "redacted.aivencloud.com:26484",
  "client_id": "a09a129aa418",
  "group_id": "a09a129aa418",
  "registry_host": "redacted.aivencloud.com",
  "registry_port": "26487",
  "host": "",
  "log_level": "INFO",
  "security_protocol": "SSL",
  "ssl_cafile": "/var/run/config/nais/ca.pem",
  "ssl_certfile": "/var/run/config/nais/service.cert",
  "ssl_keyfile": "/var/run/config/nais/service.key",
  "karapace_rest": true,
  "karapace_registry": false
}

[mortenlj]

After some more digging today, I found that this is caused by my packaging not accounting for the fact that pip wheel <karapace checkout> would fetch kafka-python from PyPI, while in requirements.txt a VCS url is used. After resolving the differences between setup.py and requirements.txt I got it working. Closing this issue.