Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from #2650

Closed
joan38 opened this issue Jun 22, 2016 · 9 comments
Closed

Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from #2650

joan38 opened this issue Jun 22, 2016 · 9 comments
Assignees
@jvican

Comments

@joan38
Copy link

joan38 commented Jun 22, 2016 •
edited

Hi,

When I update I get those looping warnings:

[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:32:19 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from apache.snapshots, using Wed Jun 22 04:59:34 BST 2016
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 04:59:34 BST 2016 and null
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 14:32:19 BST 2016 and null
[warn] Choosing drivetribe.remote for org.apache.flink#flink-parent;1.1-SNAPSHOT
[warn] Unable to reparse org.apache.flink#flink-scala_2.11;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:32:18 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:32:31 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from apache.snapshots, using Wed Jun 22 04:59:34 BST 2016
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 04:59:34 BST 2016 and null
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 14:32:31 BST 2016 and null
[warn] Choosing drivetribe.remote for org.apache.flink#flink-parent;1.1-SNAPSHOT
[warn] Unable to reparse org.apache.flink#flink-scala_2.11;1.1-SNAPSHOT from apache.snapshots, using Wed Jun 22 05:03:23 BST 2016
[warn] Sorrting results from org.apache.flink#flink-scala_2.11;1.1-SNAPSHOT, using Wed Jun 22 05:03:23 BST 2016 and null
[warn] Sorrting results from org.apache.flink#flink-scala_2.11;1.1-SNAPSHOT, using Wed Jun 22 14:32:18 BST 2016 and null
[warn] Choosing drivetribe.remote for org.apache.flink#flink-scala_2.11;1.1-SNAPSHOT
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:32:54 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from apache.snapshots, using Wed Jun 22 04:59:34 BST 2016
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 04:59:34 BST 2016 and null
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 14:32:54 BST 2016 and null
[warn] Choosing drivetribe.remote for org.apache.flink#flink-parent;1.1-SNAPSHOT
[warn] Unable to reparse org.apache.flink#flink-core;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:32:54 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from drivetribe.remote, using Wed Jun 22 14:33:04 BST 2016
[warn] Unable to reparse org.apache.flink#flink-parent;1.1-SNAPSHOT from apache.snapshots, using Wed Jun 22 04:59:34 BST 2016
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 04:59:34 BST 2016 and null
[warn] Sorrting results from org.apache.flink#flink-parent;1.1-SNAPSHOT, using Wed Jun 22 14:33:04 BST 2016 and null
[warn] Choosing drivetribe.remote for org.apache.flink#flink-parent;1.1-SNAPSHOT
...

This makes SBT unusable as it's extremely slow.

The affected versions are 0.13.11 and 0.13.10 only, it works well with 0.13.9

Cheers
@eed3si9n
Copy link
Member

SNAPSHOT is slow because to guarantee it's using the latest it needs to check all the resolvers.
You can try disabling this behavior as follows:

updateOptions := updateOptions.value.withLatestSnapshots(false)

In general I don't recommend using SNAPSHOTs unless you're doing a quick integration test locally.

@joan38
Copy link
Author

joan38 commented Jun 22, 2016

Thanks for the disable config option.

But surely there is something going wrong as it says "Unable to reparse org.apache.flink" in a warning.
Is this the expected behavior? Why did it change from 0.13.10?

Thanks

@richard77
Copy link

FYI, running into same problem with sbt 0.13.12
Any scheduled work on this issue? I've done some googling around and find more people having this issue. It hinders the development work. Disabling the use of snapshots is not always the solution..

@caoilte
Copy link

caoilte commented Nov 29, 2016

Documentation on

updateOptions := updateOptions.value.withLatestSnapshots(false)

http://www.scala-sbt.org/1.0/docs/sbt-0.13-Tech-Previews.html#Latest+SNAPSHOTs

There doesn't seem to be any downside to setting it to false if you only ever work with local SNAPSHOTs.

What I don't understand is why if this feature was introduced in 0.13.6 it didn't become a problem until 0.13.10.

@eed3si9n
Copy link
Member

eed3si9n commented Nov 29, 2016 •
edited

This setting flip flopped a bit:

@SoboLAN
Copy link

SoboLAN commented Apr 18, 2017

I have the same issue after upgrading to 0.13.12.

Are there any news about this?

jvican added a commit to scalacenter/librarymanagement that referenced this issue May 11, 2017
@jvican
Fix sbt/sbt#2650: Degrade log level and clarify
ecbb20b 
To ensure that SNAPSHOTs are always the latest, we go through all the
resolved modules and check their timestamps. Good.

However, if in the process of reparsing/redownloading the module
descriptor we fail (or it's not found in that resolver at all), then we
cannot refresh the resolved module that could have been internalized or
heavily cached in memory by ivy. We do this for correctness.

This patch does two things:

1. Adds more comments.
2. Warns only when there are parsing errors in ivy files.
3. Adds debug info in the rest of the cases.

This removes the pain of seeing `Unable to parse` that could be caused by
other reasons that are not related to parsing at all and which would not
affect the algorithm at hand. For instance, if we get a URLResource,
that's totally fine -- there is no way we could parse the ivy file. Ivy
uses URLResources in several cases where the artifact origin URL is
specified.
@eed3si9n eed3si9n added the ready label May 11, 2017
@jvican
Copy link
Member

jvican commented May 11, 2017

I've addressed this issue here: scalacenter/librarymanagement@ecbb20b. Previously, we were showing these logs for all the failures that could happen while re-fetching module descriptors for SNAPSHOTs.

My commit only warns on parser errors now, and it will add as debug information other reasons that could have happened these annoying bugs in your sbt logs: resources with other implementations, missing descriptor in resolver, etc.

For the record, the underlying snapshot resolution has not changed, but only how the logs are handled. If you still want to address the fact that SNAPSHOT resolution is slow, pick one:

  1. Disable latestSnapshots and only work locally with SNAPSHOTs.
  2. For God's sake, use a versioning scheme based on git information. Sbt-dynver (https://github.com/dwijnand/sbt-dynver) gives a good example of how it looks like.

@joan38
Copy link
Author

joan38 commented May 11, 2017 •
edited

@jvican Looks good thanks.
You can add a 3rd option that is:

  1. Use sbt-coursier because life is too short to waist in artifact fetching.

@jvican
Copy link
Member

jvican commented May 11, 2017

Yes, and the best option. But the nondeterminism of snapshots is still there. Avoid snapshots. If you can't, I would say you have a tooling problem.

eed3si9n added a commit to sbt/librarymanagement that referenced this issue May 12, 2017
@eed3si9n
Merge pull request #98 from scalacenter/issue-3051
3b55a30 
Fix sbt/sbt#2650: Degrade log level and clarify
@eed3si9n eed3si9n removed the ready label May 12, 2017
@jvican jvican mentioned this issue May 31, 2017
Merged
@hmf hmf mentioned this issue Jun 29, 2017
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jvican jvican

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@eed3si9n @joan38 @caoilte @richard77 @jvican @SoboLAN