Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cached resolution: remove transitive dependencies from callers array when using consolidated resolution #2105

Closed
edeustace opened this issue Jul 14, 2015 · 3 comments
Assignees

Comments

@edeustace
Copy link

@eed3si9n - I hope I'm describing the issue correctly - let me know..

When using withConsolidatedResolution(true) can the graph.json size be further reduced by removing transitive dependencies from the callers array? For small projects it's manageable, but for a large project the json files mushroom in size and the performance of consolidated resolution deteriorates.

Steps:

    git@github.com:edeustace/sbt-callers-sample.git
    cd sbt-callers-sample/lib-two
    sbt publish-local
    cd ../lib-one
    sbt publish-local
    cd ..
    sbt update

Then inspect the graph.json for module a. You should see something like:

     [
      {
        "caller": {
          "organization": "org.scala-sbt.temp",
          "name": "temp-resolve-1fd7120a8dac7d10ffc45ff63fc42f19d0d148e1",
          "revision": "1.0",
          //...
        },
        //...
      },
      {
        "caller": {
          "organization": "com.ee",
          "name": "lib-one_2.10",
          "revision": "0.0.1-SNAPSHOT",
          //...
        },
        //...
      },
      {
        "caller": {
          "organization": "com.ee",
          "name": "lib-two_2.10",
          "revision": "0.0.1-SNAPSHOT",
          //...
        },
      }
      //...
    ]

The tmp org (org.scala-sbt.temp) is fine as is the direct dependency lib-one.

The issue is that lib-two is listed as a caller even though it's a transitive dependency of a.

@eed3si9n eed3si9n changed the title remove transitive dependencies from callers array when using consolidated resolution cached resolution: remove transitive dependencies from callers array when using consolidated resolution Jul 16, 2015
@eed3si9n
Copy link
Member

@edeustace Thanks for the report. This case is a generalized one for #1763.
@paddymahoney reported he's seeing OOM in 0.13.9-RC2.

jsuereth added a commit that referenced this issue Jul 18, 2015
Fixes #2105/#1763. Cached resolution: removes duplicate callers during merge
@edeustace
Copy link
Author

Just ran 0.13-RC3 against the project that was grinding to a halt, it's now making use of consolidated resolution with a clean update averaging ~80s (was ~240s). Thanks @eed3si9n

@eed3si9n
Copy link
Member

@edeustace Thanks for the confirmation. Were old cached resolution grinding to the halt, or the plain update too? I'm curious about the performance comparison. Never mind you said ~240s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants