[SPARK-11272][Core][UI] Support importing and exporting event logs from HistoryServer web portal #9238
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jerryshao
changed the title
|
Test build #44197 has finished for PR 9238 at commit
|
jerryshao
changed the title
|
Test build #44221 has finished for PR 9238 at commit
|
jerryshao
changed the title
|
Updated UI:
|
|
Test build #44324 has finished for PR 9238 at commit
|
|
Jenkins, retest this please. |
|
Test build #44335 has finished for PR 9238 at commit
|
| * @throws SparkException if there's any exception read in the zip stream. | ||
| */ | ||
| @throws(classOf[SparkException]) | ||
| def readEventLogs(zipStream: ZipInputStream): Unit |
There was a problem hiding this comment.
Rather than make this abstract, can you add a stub method which just goes throw new SparkException("Unsupported Feature"). That way any existing history provider will still load
|
|
The download functionality is super cool, but the import part has somewhat confusing semantics. The history server is supposed to dynamically sync all logs in a specific directory. With this patch you can have logs that you uploaded but no longer exist, and the original logs don't have to be from the same path. Above all this seems like a rather niche use case. |
|
@jerryshao would you mind closing this PR and maybe re-opening one that only supports the download functionality? |
|
OK, I see. I will close this PR, thanks a lot for your comments. |
asfgit
pushed a commit
that referenced
this pull request
Oct 13, 2016
…toryServer UI ## What changes were proposed in this pull request? This is a reworked PR based on feedback in #9238 after it was closed and not reopened. As suggested in that PR I've only added the download feature. This functionality already exists in the api and this allows easier access to download event logs to share with others. I've attached a screenshot of the committed version, but I will also include alternate options with screen shots in the comments below. I'm personally not sure which option is best. ## How was this patch tested? Manual testing  Author: Alex Bozarth <ajbozart@us.ibm.com> Closes #15400 from ajbozarth/spark11272.
uzadude
pushed a commit
to uzadude/spark
that referenced
this pull request
Jan 27, 2017
…toryServer UI ## What changes were proposed in this pull request? This is a reworked PR based on feedback in apache#9238 after it was closed and not reopened. As suggested in that PR I've only added the download feature. This functionality already exists in the api and this allows easier access to download event logs to share with others. I've attached a screenshot of the committed version, but I will also include alternate options with screen shots in the comments below. I'm personally not sure which option is best. ## How was this patch tested? Manual testing  Author: Alex Bozarth <ajbozart@us.ibm.com> Closes apache#15400 from ajbozarth/spark11272.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch helps user to easily download the event logs from HistoryServer, or upload other's log to debug and replay. Quite useful for user to seek help or identify problem for others. Here is the screenshot:
Please review. Thanks a lot.