Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Default logging level causing issues on Spark cluster #14

Closed
dfenn opened this issue Nov 8, 2018 · 2 comments
Closed

Default logging level causing issues on Spark cluster #14

dfenn opened this issue Nov 8, 2018 · 2 comments

Comments

@dfenn
Copy link

dfenn commented Nov 8, 2018

I'm running Sherpa on a Spark cluster (I've written a little Spark scheduler), and the default logging level is causing my Spark jobs to hang.

I've fixed it for my case by changing the logging options in each of the source files to

logging.basicConfig(level=logging.WARNING)
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
@LarsHH
Copy link
Collaborator

LarsHH commented Nov 15, 2018

Great! Would you be happy to push the modifications to a branch then I can check the behavior on my side and merge it. Also, would you be happy to share your Spark scheduler?
Thanks, Lars

LarsHH pushed a commit that referenced this issue Jan 12, 2019
Issue #14: set logging level to warning
6bdc8ce
@LarsHH
Copy link
Collaborator

LarsHH commented Jan 12, 2019

Resolved: 6bdc8ce

@LarsHH LarsHH closed this as completed Jan 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@LarsHH @dfenn