Can't build schema registry #183
Comments
|
The dependency it is complaining about is Kafka, not rest-utils or common, so it sounds like those probably installed successfully. It looks like it cached a failure to find the jar, but that jar definitely exists in the maven repository. Can you try running the maven command again and add the |
|
Same error with mvn -U compile. Note, I didn't install the Confluent Kafka package, I am trying to run schema registry and rest api using mesos on Kafka, therefore I've built my own Kafka, the docs doesn't say it's reliant on that package, and the command rest-util both built successfully. |
|
Right, we don't expect you to have built Kafka yourself. Neither common nor rest-utils use Kafka, so they wouldn't run into the same error. The relevant dependency is in This uses a couple of properties defined in the root And that same file includes the necessary repository: I just removed all the data from my local maven repository and built again, and I see it correctly pulling in the dependency: Maybe there's a connectivity issue with packages.confluent.io that's causing it to record a failure? Can you download the file I just linked directly using cURL or wget? |
|
I can download that specific file with wget, but I am still getting When I try to compile. I've gone to my local local repo can removed the org.apache.kafka stuff, no dice there either. I am stumped on this. Would it help if all the 2.0-SNAPSHOT stuff was just pre built in the mvn repo? Basically I am trying to get kafka-rest built (the new versions) to help in troubleshooting some API requests in 1.0 (there are some issues fixed that may have more robust logging for me) |
|
Agreed, this seems weird. The only other thing I can think of is that the dependency has scope set as "provided", but for mvn compile that shouldn't affect anything. The 2.0-SNAPSHOT version in that message refers to the kafka-avro-serializer artifact in the schema registry, so that's what you're trying to build. Kafka and the Confluent projects are versioned differently. I don't think there's an easy fix except to get Maven to resolve the dependency properly. Is there any other downloading/downloaded log lines before this that would be relevant? kafka-avro-serializer should require kafka-clients and the kafka_2.10 jars, so you should see both (assuming a clean local repository). If you can't figure out the issue, you can also try changing the kafka.version variable to 0.8.2.1. That should pick up the Apache release from Maven Central and it should have no problem building against that version. |
|
Here is the full log: Downloading: http://packages.confluent.io/maven/org/apache/kafka/kafka_2.10/0.8.2.0-cp/kafka_2.10-0.8.2.0-cp.pom |
|
That looks like there's some issue downloading the files -- they definitely exist, but you should see corresponding "Downloaded" lines, and it's complaining about the missing POM for org.apache.kafka:kafka_2.10:jar:0.8.2.0-cp for the same reason. This seems strange if you're able to download the files directly. Any chance there's a proxy or mirror setting somewhere that -U isn't handling? e.g. if you had Nexus/Artifactory/Archiva that had cached a missing file incorrectly? In any case, changing the version in the pom.xml to 0.8.2.1 should also work. |
|
I changed it to 0.8.2.1-cp and that didn't work. I added the apache registry, and used 0.8.2.0 as my kafka version and that DID work. Will that hurt me long term? |
|
Sorry, I probably wasn't clear. When I said to try 0.8.2.1, I meant without the -cp suffix. That is the latest Apache release of Kafka. It turns out that 0.8.2.0-cp and 0.8.2.1 are exactly the same, but since 0.8.2.1 wasn't released when we released v1 of Confluent Platform, we had to tag it with a different number. Since this dependency is only pulled in during compile time (the runtime jar needs to be provided by the project using the serializers), using a different version that has a matching API should not be a problem. Since you got this working with that minor change and it seems like it's probably some issue with your environment since we couldn't track down why those downloads weren't working through maven, I'm going to close this for now. Feel free to reopen if you want to work on digging deeper if you think this might still be an issue with the schema-registry code. |
|
I experienace the same problem on my Mac with java7: Full log |
|
I tried with wget to download missing jar and found that 2.0-SNAPSHOT version aren't awailable on the maven repo. However, there's no problem accessing 1.0 jars. |
|
@satybald That's actually a different problem -- you don't have the 2.0-SNAPSHOT version of confluent-common (and if you don't have that, you also won't have the snapshot version of rest-utils, which is also required). As mentioned in the development section of the docs, you'll need to build and install both of those repositories as well -- we don't publish snapshot jars to our package repositories. |
|
@ewencp got it! thanks for this information. |
|
What's the latest confluent development version that can be pulled for Kafka rest proxy - so that the common and rest-utils, schema-registry and kafka-rest gets installed successfully with minimal steps.? |
|
I tried that out, but one of the test cases fails while trying to install the schema-registry, could you please let me know if I am missing something here. Thanks. Versions from pom.xml - T E S T SRunning io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThreadTest Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ |
|
It's possible that's timing related -- some of the tests run a real ZK + Kafka + Schema Registry setup and if your box isn't beefy enough or is overloaded with other work, they may timeout. You might check the affected test and see if increasing timeouts helps. |
|
Thanks much Ewen. |
|
I had a similar issue and it turned out to be my Maven configuration. My maven settings.xml is configured to use my company's Maven Mirror and that repository needed to have the confluent Mirror repository added in order for it to download from http://packages.confluent.io/maven. I hope this helps! |
|
So here is my problem
but it didnt help. I also tried installing the required jars one by one in my local repo using below command but i dont have patience and time to do this for all the jars F1 F1 F1 :( :( . |
I am trying to build schema registry. It says build rest-utils and common first (I did that) and now when I am building schema-registry I get this:
[ERROR] Failed to execute goal on project kafka-avro-serializer: Could not resolve dependencies for project io.confluent:kafka-avro-serializer:jar:2.0-SNAPSHOT: Failure to find org.apache.kafka:kafka_2.10:jar:0.8.2.0-cp in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]
[ERROR]
Perhaps different instructions on building?
Thanks
The text was updated successfully, but these errors were encountered: