Control characters appearing in job output in 0.6.0

[nosmo]

I'm seeing control characters show up in coloured job output where previously there were none before. It seems that this could be somewhat related to #141.

Test environment

• Jenkins Version: 2.148
• AnsiColor Version: 0.6.0
• Hostsystem: Debian stretch

Expected behavior

Up until 0.6.0 I would only occasionally see [0m characters show up and they did not affect output. The following appears when testing against 0.5.3

Actual behavior

With 0.6.0 I now see more control characters and in some cases no colour output at all:

In more complex jobs with more varied colour output, the colour output will not be properly bounded and will leak into following lines.

Steps to reproduce the behavior

The above outputs were reproduced by running the following job using the pipeline library:

def label = "terraform-${UUID.randomUUID().toString()}"

podTemplate(label: label, inheritFrom: 'jnlp', containers: [
  containerTemplate(
      name: 'terraform',
      image: 'hashicorp/terraform',
      ttyEnabled: true,
      command: 'cat',
      alwaysPullImage: true,
  )
]) {
    node(label) {
        container('terraform'){
            ansiColor('xterm') {
                sh """
                 terraform plan
                """
            }
        }
    }
}

[choppedpork]

I'm seeing the same issue - incidentally first spotted in a terraform build as well however this also happens when executing ansible-playbook jobs:

[kapoorlakshya]

Seeing the same in console output from Cucumber. Rolling back to v0.5.3 works for now. Not a 100% sure, but I think this was introduced by #132.

[dwnusbaum]

Yes this was probably introduced by #132. If anyone is able to create a PR with a minimal failing test case similar to the tests in AnsiColorBuildWrapperTest it would help us understand the core part of the problem. Might be similar to #135, so I would be interested to see if something like #137 would help.

[netsandbox]

See also https://issues.jenkins-ci.org/browse/JENKINS-55139

[dwnusbaum]

I attached a custom snapshot build of the plugin to JENKINS-55139 based on master, which includes #137. If anyone seeing the issue can test that snapshot and let me know whether it helps, that would be great.

[jrudolph]

Still broken for us with the custom version attached to JENKINS-55139. Had to downgrade to 0.5.3.

E.g. broken: https://jenkins.akka.io:8498/job/pr-validator-per-commit-jenkins/14436/console
With 0.5.3: https://jenkins.akka.io:8498/job/pr-validator-per-commit-jenkins/14437/console

[dwnusbaum]

Ok, then perhaps this is a distinct issue related to Jesse's comment here: #137 (comment). If anyone has time to create a failing test case in AnsiColorBuildWrapperTest for the issue that would make it much more likely for this to be fixed quickly.

[dwnusbaum]

Thanks for the links @jrudolph. I am not able to reproduce a failure in quite the same way unless I disable colorized output from the job's configuration page, although perhaps the appearance has degraded even further that it looked originally when you were running 0.6.0, because 0.5.3 does not understand the log format used in 0.6.0, so that might explain the discrepancy. I took a line from your logs and did some local testing to reproduce the issue.

Here are the 3 variants of the issue that I am seeing:

1. Escape sequences are completely ignored. This is what I see running ansicolor 0.6.0, building a freestyle project with colorized output disabled (this is what your logs look like, and could perhaps be reproduced by running a freestyle job in ansicolor 0.6.0 with colorized output enabled, downgrading to ansicolor 0.5.3, and viewing the log again):

2. Escape sequences are not completely ignored, but some of them show up in the logs directly and some log text is incorrectly being hidden. This is what I see running ansicolor 0.6.0, building a freestyle project with colorized output enabled:

3. Escape sequences are handled correctly. This is what I see with a local fix for variant 2 of the issue (matching the appearance with ansicolor 0.5.x):

It would be very helpful to understand if you and other users are seeing variant 1 or variant 2 for builds built when running ansicolor 0.6.0 while ansicolor 0.6.0 is still installed (not after downgrading to 0.5.3 and looking at logs for builds run under version 0.6.0). If you are seeing variant 1, then I have no idea what is going on, but if you are seeing variant 2 then I know how to fix the problem.

For anyone seeing variant 1 of the issue, are you running Freestyle or Pipeline jobs? Are there any error messages in your Jenkins logs when you view the console output for a job? If it is a freestyle job, is colorized output enabled?

[jrudolph]

It seems it's variant 2 here.

[dwnusbaum]

@jrudolph Thanks for the update, in that case I think #147 should help. I'll try to attach a snapshot build to the Jira ticket soon so anyone experiencing the issue can see if it helps.

[JoeyG1973]

I am noticing that it has something to do with the here document like stuff when you call things.

Color doesn't work here:

        stage ('Preflight') {
            steps {
                sh '''
                    pwd
                    env
                    echo "\033[34m fi fi fo fum \033[0m"
                '''

Color works like this and wierdly enough anything after the single quote echo with ansi all color kinda sorta works.... it's missing some of the closing escapes sometimes and keeps the color until it finds a close or a new color.

        stage ('Preflight') {
            steps {
                sh 'pwd'
                sh 'env'
                sh 'echo "\033[34m fi fi fo fum \033[0m"'
            }

[ababushk]

I have similar issues on Jenkins 2.150.2 and 0.6.1

[dwnusbaum]

I just released 0.6.2 with a change that should improve this issue. If you are still seeing it after the upgrade, please copy-and-paste some of the broken output from Jenkins here (preferably replacing literal occurrences of the ASCII character 0x1b with the text 0x1b so we can see what is happening to the escape sequences).

[ababushk]

@dwnusbaum it works like a charm, thank you!

[kapoorlakshya]

Hi @dwnusbaum, the fix in v0.6.2 is only partially working when using Cucumber. What I mean by partially is that I still see the control characters and no colors in the console output as the text shows up. However, when I refresh the page the control characters disappear and the text shows up in color.

This behavior is not experienced in v0.5.3. Also, this only happens in the console output and not in the Blue Ocean view.

Before refresh:

After page refresh:

Raw output:

�[32mAnd I access the '�[32m�[1mSearch�[0m�[0m�[32m' �[32m�[1mtab�[0m�[0m�[32m�[90m

Cucumber command:

bundle exec cucumber -t @search_page -f json -o cucumber.json -f pretty --color --expand

[fgnathan]

I'm seeing that behaviour as well. It seems that if I load the Console Output page before any escape codes have appeared, I'll see the codes. Reloading the page, or loading it after some colour output has appeared will show the colours properly.

[dwnusbaum]

@kapoorlakshya @fgnathan That sounds like #136, but I don't have any ideas about the cause. It would be great to add any additional reproduction information or diagnostics to that issue (perhaps it is related to size of the logs, whether the logs are truncated, etc.) to help us understand what is happening.

[ulich]

I also see this issue

It happens with very little amount of logs, so no truncating.
In our case, there are commands in the pipeline definition before, that do not have any ANSI color output.

Maybe that helps

[dwnusbaum]

Thanks for the info @ulich, but let's try to keep any discussion about that issue in #136 for clarity.

[kapoorlakshya]

@dwnusbaum Thanks! I will add all relevant information in that issue.

[fearnycompknowhow]

It seems that Terraform attempts to format the output using color. However, Jenkins does not output this formatting correctly and thus you see those unusual characters in the output.

Thanks to this thread, I was able to make Terraform omit those characters in the output by using the -no-color flag.

This options is available for at least three of the major Terraform functions: show, plan, and apply. It's probably available for others, I just haven't confirmed it.