Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZEPPELIN-2815] Improve interpreter dependencies logging #2507

Closed
wants to merge 3 commits into from

Conversation

necosta
Copy link
Contributor

@necosta necosta commented Jul 27, 2017

What is this PR for?

As a developer, I want to be able to debug issues on dependency resolution "easier".
To achieve this, making available (in debug mode), existing listeners.
Also, exposing exceptions on dependency resolver.

What type of PR is it?

Improvement

What is the Jira issue?

How should this be tested?

Enable DEBUG logging mode. Import dependencies. Check detailed info. from dependency loading being logged.

Questions:

  • Does the licenses files need update? N
  • Is there breaking changes for older versions? N
  • Does this needs documentation? N

@necosta
Copy link
Contributor Author

necosta commented Jul 27, 2017

Hi @Leemoonsoo and others, should be a trivial one. Low priority. Many thanks

@@ -122,7 +120,7 @@ public void transferSucceeded(TransferEvent event) {
@Override
public void transferFailed(TransferEvent event) {
transferCompleted(event);
event.getException().printStackTrace(out);
logger.warn(event.getException().getMessage());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warn and not error as it might be legit to not find a dependency in one repo (should fallback to the next)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good update

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd convert to logger.warn(message, event.getException()) here
https://www.slf4j.org/api/org/slf4j/Logger.html#warn(java.lang.String,%20java.lang.Throwable)

@@ -177,8 +177,9 @@ public synchronized void copyLocalDependency(String srcPath, File destPath)
DependencyFilterUtils.andFilter(exclusionFilter, classpathFilter));
try {
return system.resolveDependencies(session, dependencyRequest).getArtifactResults();
} catch (NullPointerException ex) {
throw new RepositoryException(String.format("Cannot fetch dependencies for %s", dependency));
} catch (Exception ex) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be still NPE? since Exception can cache more exception types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I changed it to Exception as I see that org.sonatype.aether.RepositorySystem can throw DependencyResolutionException and I suspect we can have file exceptions and other in the whole org.sonatype.aether library. But because I didn't actually get an error here, I'll leave the final decision to you. Should we revert this to only have NPE or keep it broad with Exception?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think more specific exception would be preferred...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now

catch (NullPointerException | DependencyResolutionException ex)

@@ -55,13 +53,13 @@ public void transferInitiated(TransferEvent event) {
@Override
public void transferProgressed(TransferEvent event) {
TransferResource resource = event.getResource();
downloads.put(resource, Long.valueOf(event.getTransferredBytes()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need converting? (LINE 62 as well)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. It's already a long

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both are already a long I should say

@@ -177,8 +177,9 @@ public synchronized void copyLocalDependency(String srcPath, File destPath)
DependencyFilterUtils.andFilter(exclusionFilter, classpathFilter));
try {
return system.resolveDependencies(session, dependencyRequest).getArtifactResults();
} catch (NullPointerException ex) {
throw new RepositoryException(String.format("Cannot fetch dependencies for %s", dependency));
} catch (Exception ex) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think more specific exception would be preferred...

@@ -135,10 +133,10 @@ private void transferCompleted(TransferEvent event) {

@Override
public void transferCorrupted(TransferEvent event) {
event.getException().printStackTrace(out);
logger.error(event.getException().getMessage());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logger.error("some message", event.getException()); instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@necosta
Copy link
Contributor Author

necosta commented Aug 7, 2017

@1ambda , @felixcheung , ready for another round of review. Thanks

Copy link
Member

@felixcheung felixcheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG except for 1 comment

@@ -122,7 +120,7 @@ public void transferSucceeded(TransferEvent event) {
@Override
public void transferFailed(TransferEvent event) {
transferCompleted(event);
event.getException().printStackTrace(out);
logger.warn(event.getException().getMessage());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd convert to logger.warn(message, event.getException()) here
https://www.slf4j.org/api/org/slf4j/Logger.html#warn(java.lang.String,%20java.lang.Throwable)

@necosta
Copy link
Contributor Author

necosta commented Aug 8, 2017

Makes sense @felixcheung . Changed warn message to
logger.warn("Unsuccessful transfer", event.getException());

@necosta necosta closed this Aug 8, 2017
@necosta necosta reopened this Aug 8, 2017
@Leemoonsoo
Copy link
Member

LGTM and merge to master if no further discussions.

@asfgit asfgit closed this in 26a39df Aug 24, 2017
@liuhengzhi liuhengzhi deleted the zeppelin2815 branch July 3, 2018 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants