Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-48344] Lightweight timestamp annotations #25

Merged
merged 16 commits into from Feb 7, 2019

Conversation

@jglick
Copy link
Member

jglick commented Sep 28, 2018

JENKINS-48344

Uses jenkinsci/workflow-job-plugin#105 (and its upstreams) to implement much cheaper timestamping for Pipeline builds. Produces log file output like

…
[2018-09-28T22:49:03.570Z] ^[[8mha:////etc.etc.^[[0m[Pipeline] sh
[2018-09-28T22:49:03.895Z] [p] Running shell script
[2018-09-28T22:49:04.427Z] + sleep 1
[2018-09-28T22:49:05.249Z] + sleep 1
[2018-09-28T22:49:06.262Z] + sleep 1
[2018-09-28T22:49:07.281Z] + sleep 1
[2018-09-28T22:49:08.298Z] + sleep 1
…

which gets rendered like

screenshot

No ConsoleNotes needed.

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Sep 28, 2018

BTW this does not need #24. It could benefit from jenkinsci/jenkins#3662.

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Sep 28, 2018

Some LogFileReaderTest regressions, probably just caused by updating some dep. Can look at it later.

Copy link
Member Author

jglick left a comment

Oh and have not yet checked impact on

  • per-step logs
  • Blue Ocean
long buildStartTime = build.getStartTimeInMillis();
String html = text.toString(true);
int start;
if (html.startsWith("<span class=\"pipeline-new-node\" ")) { // cf. LogStorage.startStep

This comment has been minimized.

Copy link
@jglick

jglick Sep 28, 2018

Author Member

Unpleasant that this is necessary. Not sure if it is solvable upstream. You would probably expect that the MarkupText is what is produced before this span is added by FileLogStorage.

@jglick jglick changed the title [JENKINS-48344] Lightweight timestamp annotations [WiP] [JENKINS-48344] Lightweight timestamp annotations Sep 28, 2018
jglick added 7 commits Oct 4, 2018
…th other tests.
@jglick jglick changed the title [WiP] [JENKINS-48344] Lightweight timestamp annotations [JENKINS-48344] Lightweight timestamp annotations Oct 16, 2018
@jglick jglick changed the title [JENKINS-48344] Lightweight timestamp annotations [WiP] [JENKINS-48344] Lightweight timestamp annotations Oct 16, 2018
@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Oct 16, 2018

Per-step display in BO shows timestamps, since BO completely ignores ConsoleAnnotator and ConsoleNotes, but at least they are sort of human-readable:

screenshot

@jglick jglick changed the title [WiP] [JENKINS-48344] Lightweight timestamp annotations [JENKINS-48344] Lightweight timestamp annotations Oct 16, 2018
Copy link
Member

dwnusbaum left a comment

Overall approach looks good to me, just added some java.time suggestions. The Blue Ocean view seems kind of awkward, do you think there is any easy way to clean it up? CC @vivek to see what he thinks of how it looks in Blue Ocean.

<java.level>7</java.level>
<revision>1.8.11</revision>
<changelist>-SNAPSHOT</changelist>
<jenkins.version>2.121.1</jenkins.version>

This comment has been minimized.

Copy link
@dwnusbaum

dwnusbaum Oct 22, 2018

Member

Quite a jump, although required for the JEP-210 updates. Here are stats for the plugin's usage by Jenkins core version: http://stats.jenkins.io/pluginversions/timestamper.html. According to those stats, 89.63% of users of timestamper-1.8.10 are on Jenkins 2.121.1 or newer, so this looks reasonable to me.

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Oct 24, 2018

The Blue Ocean view seems kind of awkward, do you think there is any easy way to clean it up?

Only by having Blue Ocean implement a custom display hook, like it did for ANSI coloration. Since BO ignores ConsoleNote and ConsoleAnnotatorFactory, without this patch there is no display of timestamps—so presumably BO-focused installations would not have been using this plugin to begin with. To my knowledge the BO rendering system is simply not extensible via plugins the way the classic UI is.

…he middle of a line.
Write timestamps as a single byte[] to avoid interleaving in some cases
@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Jan 16, 2019

Also expected to solve JENKINS-54081 and JENKINS-55257.

@timja
timja approved these changes Jan 19, 2019
Copy link
Member

timja left a comment

@StevenGBrown Would you be able to take a look at this please?

@olamy
olamy approved these changes Jan 19, 2019
Copy link
Member

olamy left a comment

LGTM

@casz
casz approved these changes Jan 25, 2019
Copy link
Member

casz left a comment

LGTM

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Jan 28, 2019

I have requested comaintainership of this plugin. According to policy, there is a two-week waiting period.

@jglick jglick merged commit 0e2fe42 into jenkinsci:master Feb 7, 2019
2 checks passed
2 checks passed
continuous-integration/jenkins/incrementals Deployed to Incrementals.
Details
continuous-integration/jenkins/pr-merge This commit looks good
Details
@jglick jglick deleted the jglick:lightweight-annotator-JENKINS-48344 branch Feb 7, 2019
jglick added a commit to jglick/workflow-job-plugin that referenced this pull request Feb 7, 2019
@jglick jglick mentioned this pull request Feb 7, 2019
@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Feb 7, 2019

Possible casualty (untested): TimestamperAPI.read, used by at least aws-cloudwatch-logs-publisher (though that now has an alternative in pipeline-cloudwatch-logs anyway).

@dankirkd

This comment has been minimized.

Copy link

dankirkd commented Feb 8, 2019

@jglick Can you indicate how to use the plugin in a Jenkinsfile via the "global option", as opposed to having to wrap blocks within each of the "steps"? I can't find any example of that, and have tried to get an answer from @StevenGBrown on https://wiki.jenkins.io/pages/viewpage.action?pageId=46334448 with no success.

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Feb 8, 2019

how to use the plugin in a Jenkinsfile via the "global option"

You do not—it is global configuration (whether specified in the UI or configuration-as-code or whatever).

@dankirkd

This comment has been minimized.

Copy link

dankirkd commented Feb 8, 2019

What is the default setting on the "all pipelines" config? Is it set by default?

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Feb 8, 2019

What is the default setting on the "all pipelines" config? Is it set by default?

No, it is off by default, to minimize the behavioral change caused by merely updating the plugin.

@alde

This comment has been minimized.

Copy link

alde commented Feb 18, 2019

It seems to me that hitting /timestamps/?time=%27HH:mm:ss%27&appendLog no longer has any effect, as the result is identical to hitting /consoleText.

Could this be a side-effect of this change?

@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Feb 18, 2019

@alde quite possibly. I am unaware of that URL pattern or what it is supposed to do.

@dankirkd

This comment has been minimized.

Copy link

dankirkd commented Feb 18, 2019

@alde Try /timestamps/?time=HH:mm:ss&appendLog as that seems to work fine for me with version 1.8

@jglick It is just outputs the timestamp and with appendLog appends the text that goes with it - effectively looks like consoleText with a timestamp.

@alde

This comment has been minimized.

Copy link

alde commented Feb 19, 2019

With version 1.9 nothing I put in time={} has any effect anymore. I'll look at whether downgrading to 1.8 will have an effect. Anyhow, this may be an indication of a regression.
In 1.9 both /timestamps/?time=HH:mm:ss&appendLog and /consoleText output lines like

[2019-02-18T11:40:12.626Z] [WS-CLEANUP] Deleting project workspace...
@jglick

This comment has been minimized.

Copy link
Member Author

jglick commented Feb 19, 2019

I see, there are some scripting URLs which apparently have no test coverage. As noted in #25 (comment) there is also a Java API which I think has no test coverage. Probably needs some follow-up work to write proper test cases for all these things and make any adjustments needed for them to work with the new architecture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.