This repository has been archived by the owner on May 2, 2023. It is now read-only.
Unable to install kafka-connect-datagen:0.1.0 #654
Comments
|
From the command line (i.e., outside of Docker), what is the output from the following command? curl https://api.hub.confluent.io/api/plugins |
|
@mstolin which branch do you use? |
|
@gAmUssA |
cp-all-in-one(5.0.1-post): docker-compose build --no-cache connect
Building connect
Downloading context: https://github.com/confluentinc/kafka-connect-datagen/raw/master/Dockerfile-confluenthub 778B
Step 1/3 : FROM confluentinc/cp-kafka-connect:5.0.0
---> 7df8759460f7
Step 2/3 : ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"
---> Running in 231cd9016b9f
Removing intermediate container 231cd9016b9f
---> bac74ec192f8
Step 3/3 : RUN confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:0.1.0
---> Running in b6ba9e31bb96
Running in a "--no-prompt" mode
Implicit acceptance of the license below:
The Apache License, Version 2.0
https://www.apache.org/licenses/LICENSE-2.0
Downloading component Kafka Connect Datagen 0.1.0, provided by Confluent, Inc. from Confluent Hub and installing into /usr/share/confluent-hub-components
Adding installation directory to plugin path in the following files:
/etc/kafka/connect-distributed.properties
/etc/kafka/connect-standalone.properties
/etc/schema-registry/connect-avro-distributed.properties
/etc/schema-registry/connect-avro-standalone.properties
Completed
Removing intermediate container b6ba9e31bb96
---> 42306358551a
Successfully built 42306358551a
Successfully tagged confluentinc/kafka-connect-datagen:0.1.0 |
|
|
If you're willing, here is one more suggestion to try: instead of building the connector image locally, run the CP connect image and then manually install the connector.
image: confluentinc/kafka-connect-datagen:0.1.0
build:
context: https://github.com/confluentinc/kafka-connect-datagen/raw/master/Dockerfile-confluenthub
dockerfile: Dockerfile-confluenthuband replace it with the following line image: confluentinc/cp-kafka-connect:5.0.0
|
|
@ybyzek I'm following the suggestion you made on #646 to come here and try your suggestion, above. After editing the docker-compose.yml file as suggested, here are the results: Also, the ksql-datagen container failed to start. The tail of the log contains: I'm not sure if it is related to the first error. |
|
Thanks @hpinsley . The error from the |
|
I'll try that. Does it matter that your solution for getting connect to come up uses an older image? My main objective is to try out KSQL and KTables. |
|
@hpinsley : by "older image", I believe you mean this line, correct? image: confluentinc/cp-kafka-connect:5.0.0It shouldn't matter, but to rule it out as a possibility, you could try image: confluentinc/cp-kafka-connect:5.1.0 |
|
I updated to the latest version, applied your edits, and get similar results with respect to ksql-datagen... This is what the state is: Here is just a docker ps so you can see the image versions I'm running to ensure I didn't make a mistake? And the install command failed as you predicted (although now with a slightly different error)? Is ksql-datagen a required component to try out KSQL and KTables? Thanks for your help, btw. |
But, if you are relying on
git fetch
git checkout 5.1.0-post
git pull
|
|
Argh... You're right. I'm dyslexic (pulled 5.0.1-post)! Pulled the right one. For the record, the tail of the ksql-datagen log now shows: At least now it says 5.1.0.jar ;-) Still have the problem with the install you gave me. I am indeed on Windows but I'm running everything from the Linux containers within Docker for Windows. Perhaps it is a proxy issue? |
|
@hpinsley , thank you for reporting the issue regarding the As for the issue with connecting to Confluent Hub, my colleague has a similar environment to yours (Ubuntu 18.04 LTS on Windows 10, using Docker version 18.09.0, build 4d60db4 ) and it works for him. |
|
You may also download the connector directly from Confluent Hub: https://www.confluent.io/connector/kafka-connect-datagen/ |
|
Has there been any change regarding the failure to connect? I ran into this issue while following the quickstart guide on Confluent. I am using version 5.2.1-post on Ubuntu 16.04.5 VM and running into an issue when performing |
|
Hi. I am having the same problem. Am trying to run it on docker for windows. |
|
For what it's worth, I've ran into a similar issue on MAC OS version 10.14.3. The issue appears to be directly related to attempting to install datagen.
|
|
So a weird series of events just happened. As suggested, I manually downloaded the zip file into my downloads folder using https://www.confluent.io/connector/kafka-connect-datagen/#download. I then made a new folder in confluent-5.2.1/share called "confluent-hub-components". Then, just for fun I tried rerunning I dunno if it's an issue with the folder not being there or if by manually downloading the thing I generated a valid certificate for the CLI to piggyback off of, but hopefully this provides some helpful info for resolvers (or at least a viable workaround). |
|
I tried @sadehart work around above but it did not resolve the error when trying to install kafka-connect-datagen using confluent-hub. Any other suggestions? |
@ybyzek Hey i am facing same issue, Error: Security issues** Is it open source and allowed to use along with community edition? or its allowed for enterprise edition only? |
|
@ckurdekar -- what's the machine/OS version? |
|
Hi, I'm on a macOS Mojave 10.14.5 git clone https://github.com/confluentinc/cp-docker-images I get Creating network "cp-all-in-one_default" with the default driver Error: Unknown error Thoughts? Any ideas? |
|
@bdoyle807 to isolate the issue, one suggestion is to try running the Once you download the CLI, you can try the command below from your host machine to see if it generates the same error: confluent-hub install confluentinc/kafka-connect-datagen:latest --component-dir .Would you be willing to try this? |
|
Hi, I am on CentOS Linux release 7.4.1708, Java version is openjdk version "1.8.0_181". After running I hava tried for many times I also tried |
|
@codemayq can you try the suggestion in #654 (comment) to isolate the issue to Docker or not. |
|
@ybyzek The SSL issue appears to be gone but now this: Error: Invalid options or arguments |
|
@bdoyle807 , it appears we need to specify some dirs. I have updated the comment above and copying here, this is what should be run (note the addition of confluent-hub install confluentinc/kafka-connect-datagen:latest --component-dir .It will still error out, however, after running it, you should see the download: /tmp: confluent-hub install confluentinc/kafka-connect-datagen:latest --component-dir .
Component's license:
Apache License 2.0
https://www.apache.org/licenses/LICENSE-2.0
I agree to the software license agreement (yN) y
Downloading component Kafka Connect Datagen 0.1.3, provided by Confluent, Inc. from Confluent Hub and installing into .
Unable to detect Confluent Platform installation. Specify --component-dir and --worker-configs explicitly.
Error: Invalid options or arguments
/tmp: ls confluentinc-kafka-connect-datagen
total 4
drwxr-xr-x 7 yeva wheel 224 Jun 6 08:32 .
drwxrwxrwt 46 root wheel 1472 Jun 6 08:32 ..
drwxr-xr-x 3 yeva wheel 96 Jun 6 08:32 assets
drwxr-xr-x 7 yeva wheel 224 Jun 6 08:32 doc
drwxr-xr-x 7 yeva wheel 224 Jun 6 08:32 etc
drwxr-xr-x 20 yeva wheel 640 Jun 6 08:32 lib
-rw-r--r-- 1 yeva wheel 1380 Jun 6 08:32 manifest.json
/tmp: |
with
it can be completed. |
|
@codemayq so it seems this is something related to the specific Docker environment. Short-term workaround: manually mount those jars into the connect Docker container via Long-term fix: we'll need to investigate further |
|
Another hypothesis is there may an issue related to a possible proxy interfering with SSL certs. If the users on this GH thread have the availability to do so, here are some additional points to check:
Server certificate
subject=/C=US/ST=CA/L=Palo Alto/O=Confluent, Inc./OU=Information Technology/CN=*.confluent.io
issuer=/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
curl -iv https://api.hub.confluent.io/api/plugins
|
|
@ybyzek the following output from the ssl call, i am on a vpn with a proxy to get out echo | openssl s_client -connect api.hub.confluent.io:443 -showcerts
CONNECTED(00000006)
write:errno=54
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Start Time: 1560182347
Timeout : 7200 (sec)
Verify return code: 0 (ok)
--- |
@bdoyle807 given the above, perhaps this user's comment would be applicable to your environment? Set the proxy settings as shown here: #731 (comment) |
|
I have the same problem，and add the dns to resolv.conf below: |
|
I'm getting the same |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
After running
docker-compose up -dinexamples/cp-all-in-onei get the following error:I was able to avoid this error after i set
connectback toimage: confluentinc/cp-kafka-connect:5.0.0.The text was updated successfully, but these errors were encountered: