New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transitive dependencies not resolved #523

Closed
kgignatyev opened this Issue Dec 9, 2016 · 18 comments

Comments

Projects
None yet
5 participants
@kgignatyev

kgignatyev commented Dec 9, 2016

we have only Maven repo and dependencies do not get resolved

@ import $ivy.`com.amazonaws:aws-java-sdk-s3:1.11.53`
@ import com.amazonaws.services.s3.AmazonS3Client
val s3c = new AmazonS3Client()

causes compile exception

Compiling ammtest.scala
Class com.amazonaws.SdkClientException not found - continuing with a stub.
Class com.amazonaws.AmazonServiceException not found - continuing with a stub.
Class com.amazonaws.AmazonWebServiceClient not found - continuing with a stub.
Class com.amazonaws.SdkClientException not found - continuing with a stub.
Class com.amazonaws.AmazonServiceException not found - continuing with a stub.
Class com.amazonaws.ClientConfiguration not found - continuing with a stub.
Class com.amazonaws.auth.AWSCredentialsProvider not found - continuing with a stub.
Class com.amazonaws.auth.AWSCredentials not found - continuing with a stub.
Compilation Failed

maven resolves dependencies fine and script works if I import all the dependencies reported by
mvn dependency:list
individually

@tkroman

This comment has been minimized.

Contributor

tkroman commented Dec 10, 2016

@kgignatyev this works for me with the latest version (which was released with probably relevant fixes earlier today). Can you re-check? In case it works for you, feel free to close the issue.

@kgignatyev

This comment has been minimized.

kgignatyev commented Dec 11, 2016

Just checked with 0.8.1, same - transitive dependencies are not resolved, please see the attached s3.scala file
s3.scala.zip

@tkroman

This comment has been minimized.

Contributor

tkroman commented Dec 11, 2016

Weird, still works for me even after local cache clean.

What do you mean by "we have only Maven repo"?

@kgignatyev

This comment has been minimized.

kgignatyev commented Dec 11, 2016

I mean that there is no ivy dependency definitions there, just standard Nexus repository server (https://www.sonatype.com/download-oss-sonatype ) but now at home (no servers) it still does not work for me.

Did you try to run file I have attached?

@tkroman

This comment has been minimized.

Contributor

tkroman commented Dec 11, 2016

Yes, I did and here's the log: https://gist.github.com/tkroman/6cfa9e9a1f0974e73cfdc7d32985ab7d

Can you clean your ivy cache and try again?

@kgignatyev

This comment has been minimized.

kgignatyev commented Dec 11, 2016

Cleared cache, same behavior:
image

@kgignatyev

This comment has been minimized.

kgignatyev commented Dec 11, 2016

OK, found how to reproduce this bug: .ivy2 and .m2 caches need to be deleted in order for $ivy to start resolving transitive dependencies. When fully populated .m2 cache exists and .ivy2 does not then transitive resolution does not work.
Please try to use project to reproduce the problem
ammonite.zip

Steps:

  1. remove or move .m2 and .ivy2
  2. cd ammonite; mvn install
  3. amm src/s3.scala
@tkroman

This comment has been minimized.

Contributor

tkroman commented Dec 11, 2016

Confirmed. I'll look into this when I'll have spare time, which would probably in a couple of days.

@Miuler

This comment has been minimized.

Miuler commented Feb 7, 2017

I have the same problem, example:

import $ivy.`com.mashape.unirest:unirest-java:1.4.5`


import com.mashape.unirest.http.Unirest

def token(user:String, password:String) = Unirest
    .post("https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyPassword?key=AIzaSyBT95pa2tui4uzyarY2fLAnpH0pqaLLfZU")
    .header("accept", "*/*")
    .header("content-type", "application/json")
    .header("Host", "www.googleapis.com")
    .header("accept-encoding", "gzip, deflate, br")
    .header("accept-language", "es,en-US;q=0.7,en;q=0.3")
    .header("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0")
    .header("x-client-version", "Firefox/JsCore/3.3.0")
    .header("accept", "*/*")
    .header("Connection", "keep-alive")
    .header("referer", "http://ec2-52-37-92-176.us-west-2.compute.amazonaws.com:8081/firebase")
    .header("origin", "http://ec2-52-37-92-176.us-west-2.compute.amazonaws.com:8081")
    .body(s"""{ "email":"$user", "password":"$password", "returnSecureToken":true }""")
    .asJson()

token("user@gmail.com", "1234") 
Class org.apache.http.HttpResponse not found - continuing with a stub.
Compilation Failed

Can not solve: Class org.apache.http.HttpResponse not found - continuing with a stub. Compilation Failed

@kgignatyev

This comment has been minimized.

kgignatyev commented Feb 8, 2017

@tkroman Could you hint what needs to be done to implement Maven resolver? I might have some time to look into it?

@tkroman

This comment has been minimized.

Contributor

tkroman commented Feb 9, 2017

@kgignatyev yeah, I was too optimistic about it. The problem is, Ammonite generates its own wrapper pom for dependencies and somewhere in there something gets lost. I spent some time debugging the hell out of ivy's internals, and even googled some similar problems, but their solutions didn't work and now I don't have enough time to do this. Even if I had, I think it won't be worth it seeing as there are plans to move from that IvyThing to coursier, which would be definitely a good thing.

@kgignatyev

This comment has been minimized.

kgignatyev commented Feb 9, 2017

Are you saying that it is too hard to implement dedicated Maven based resolver? I would be very happy if I could use only it to resolve dependencies and do not have any Ivy in sight ;)

@tkroman

This comment has been minimized.

Contributor

tkroman commented Feb 9, 2017

No, the problem is that there are some obscure resolver configuration flags (originally I thought that something like this can help: https://stackoverflow.com/questions/20799578/cant-get-ivy-to-resolve-transitive-dependencies-of-modules-in-my-local-reposito, but it didn't (or I did something wrong)), and the docs are really scarce

I also tried some workarounds, but can't really remember if anything worked. Unfortunately, I don't think I can tell you anything useful :)

@kgignatyev

This comment has been minimized.

kgignatyev commented Feb 10, 2017

@kgignatyev

This comment has been minimized.

kgignatyev commented Feb 20, 2017

I was experimenting a bit and it seems to be easy enough to maven resolver, please have look at the diff attache. basically I have modified trait LoadJar by adding maven() method that is 3 lines of code thanks to ShringWrap library https://github.com/shrinkwrap/resolver#introduction-to-shrinkwrap-resolvers
so, in the shell I can do
interp.load.maven("com.amazonaws" % "aws-java-sdk-s3" % "1.11.93")
and all seems to work.

Please have a look if that can be used, at least it seems like a good basis to address #524

shrinkwrap.diff.zip

image

@deusaquilus

This comment has been minimized.

deusaquilus commented Mar 21, 2017

Okay, here's a PR for @kgignatyev's solution, I also added a test. Please have a look and tell me if anything needs to be changed.

Edit: Not sure why noLongLines test is failing, don't see this in my local environment. Any guidance would be appreciated.
Edit: I think that jboss is bringing along a whole plethora of transitive dependencies that cause lots of nastiness. Need to investigate further.

@deusaquilus

This comment has been minimized.

deusaquilus commented Mar 22, 2017

Okay, fixed up the long-line thing and added a merge strategy so aether works properly but not sure what the Scala 2.10.6 issue is. Would appreciate any comments.

In case I haven't made it clear before, this change allows you to do:
import $maven.`com.amazonaws:aws-java-sdk-s3:1.11.53`
and it works properly with your ~/.m2/settings.xml mirrors etc...

@lihaoyi

This comment has been minimized.

Owner

lihaoyi commented May 18, 2017

The switch to coursier happened here 662859f

Hopefully this should fix it, but regardless, any contribution is going to have to be reworked.

I'm still not exactly sure what the problem is, because the samples provided work out-of-the-box on my machine, so if it's still broken someone is going to have to explain it to me in a way I can understand...

@lihaoyi lihaoyi closed this May 18, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment