-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Command with --file
considered successful even when all tests fail, JSON error appended malforms data
#1844
Comments
Hi @polarathene, I can't really comment on most of this issue. I'm not familiar with I am not a JSON expect, but I can't find any way to append the error message to an existing JSON file that doesn't result in invalid JSON. Whenever I have tried appending new JSON output to an existing JSON file, the JSON checker I have used has flagged as an error the presence of additional data after the contents of the original file. One solution would be what I tried in #1398, but that PR would simply prevent anything from being written to already existing JSON file. That would prevent the JSON file from being corrupted, but then there would be no indication of the error in the JSON file. The only alternative I can think of would be to stop scanning entirely. testssl.sh already does this in the case that the command line contains an error. For example, if testssl.txt contained:
then testing would stop as soon as the second line was parsed. The child process would for the second line would note that the command line contained an invalid option, print out the @drwetter : At the moment, if |
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit. Another issue that this commit addresses is that sometimes when parse_cmd_line() prints an error messsage and then calls help() the error message is send to stderr and other times it is sent to stdout. This commit sends all of these error messages to stderr.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit. Another issue that this commit addresses is that sometimes when parse_cmd_line() prints an error messsage and then calls help() the error message is send to stderr and other times it is sent to stdout. This commit sends all of these error messages to stderr.
I guess in this case it depends what is expected outcome if some tests fail. If you're using this for testing such as CI, if a failure is an issue, you perhaps want any failure to raise a failure regardless of some tests passing? I could understand if someone else is instead doing automated scheduled monitoring and would want to individually check the results of each and not have a failure short-circuit the tests or imply failure if 1-2 tests go wrong. Perhaps the right approach for that is some command flag to alter the behaviour, eg
Is the modified JSON actually invalid? Do you have a diff of the modification applied? I contributed this little script to a project that uses
It looks like you're already doing something that from a quick glance over the JSON snippet I provided, just the value you're updating is invalid due to ],
"scanTime" : 208
}
{
"id" : "scanProblem",
"severity" : "FATAL",
"finding" : "non-empty 'port_587.json' exists. Either use '--append' or (re)move it"
} ],
"scanTime" : "Scan interrupted"
}
{
"id" : "scanProblem",
"severity" : "FATAL",
"finding" : "non-empty 'port_587.json' exists. Either use '--append' or (re)move it"
} ],
"scanTime" : "Scan interrupted"
} It looks like you actually wanted to update
That last option would probably work in my case, but as mentioned earlier in this response, may be undesirable for some users. You could have a I'd still prefer more consistency with the expectation of using
While the JSON issue would ideally also be resolved, being able to get this behaviour would be great if it'd resolve the issue with detecting the error/failure. It seems that'll be resolved with #1871 ? |
I would not like to rewrite JSON output. I consider JSON in the sense of processing through testssl.sh more like a stream: once we passed through the stream we don't want to move back the pointer. Otherwise I am afraid we're struggling against making JSON valid again under several scenarios. It is already a headache and opening another box of pandora doesn't feel right to me. Can't you just invoke everything differently @polarathene ? I am still in the process making my mind up whether a general termination is good or whether it is not. We need some arguing here... |
That's fine... but you shouldn't be appending an error like shown above in this issue where it makes the JSON document invalid and can effectively "corrupt" it to parsers. Instead if a user needs be able to check the status of each run with I don't need tests to fail early, but it would be nice to know what failed, or if there was a failure (it's automated testing, so I don't mind if it's going to run all the other tests and return an error code at the end because one or more failed, if that comes with some output of what failed, awesome :)
That's ok I understand that as you've got a DIY approach for JSON instead of leveraging a third-party dependency as an opt-in way to get better JSON support via something like I get that
I thought I was doing individual runs for each port separately, but it seems that I kept the I do reference this issue stating
I would think failure consistency between a single call and batch via Terminating early is only useful for automated tests like in CI? (since for PRs we need to wait on results and it takes a few minutes, for some setups CI minutes might be billable too). I would vote for a flag to explicitly request to fail fast ( |
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit. Another issue that this commit addresses is that sometimes when parse_cmd_line() prints an error messsage and then calls help() the error message is send to stderr and other times it is sent to stdout. This commit sends all of these error messages to stderr.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
As suggested in drwetter#1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes. Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it. There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
Merged: #1871 (comment) Not clear if there was an option / flag to provide for that logic. If given some context on using that to test against this issue, I'll give a try 👍 |
option/flag: nope. Other than that: @dcooper16 can better answer than myself. |
Their earlier comment above ends with:
Which seems to be what the PR addressed? I can understand none of us wanting to wade through the entire discussion above 😆 Since no one else has chimed in with this problem and I have had a workaround implemented since reporting, it's not worth your valuable time. I'll leave the issue open in the mean-time for visibility, in case anyone else experiences it. |
Command line / docker command to reproduce
testssl.sh
command:testssl.sh --file /testssl.txt --mode parallel
Docker command:
testssl.txt
:Errors are not caught like a single test is when using
--file
, which was expected, at least with--mode serial
that some indication of a failure could be detected when using testing tools such asbats
.JSON output is also incorrectly appended when errors occur, making the data invalid preventing successful query as an alternative failure detection.
More details
The command works well if the json files don't already exist, but unlike a single test (instead of
--file
), the error doesn't seem to get caught by testing toolbats
'srun
method (runs command following it via a subshell). The output snippet from bats failure:This indicates that
bats
considered the previousrun
withtestssl.sh
successful, despite the error. Presumably becausetestssl.sh
handled multiple tests differently by possibly running each test in their own subshells and not bubbling up failure?Errors for each were output to console when testing without
bats
involved, each had the same in output error like when running a single test. I assume the given docker command was considered successful despite at least one (or all) tests returning an error/failure.The next
bats
test following thetestssl.sh
command is ajq
command that extracts thecipherorder_TLSv1_1
(by matching id)finding
value:The failure from that test is related to
testssl.sh
appending invalid JSON to the json output (multiple runs):Note how it appends an object
scanProblem
with],
and}
closing tags lacking matching opening tags creating invalid JSON. Otherwise I could workaround the lack of error bubbling up fortestssl.sh
by checking forscanProblem
viajq
.Expected behavior
When running multiple tests via
--file
, there should be a way to allowtestssl.sh
to bubble up the failure like a single test (without--file
) behaves, especially when run with--mode serial
. This would be helpful when usingtestssl.sh
with test suites.Existing JSON file either should not be modified (toggle via flag?) or not append content that causes malformed JSON data, since that cannot be reliably parsed to detect the failure.
Your system (please complete the following information):
Ubuntu 20.10
(Docker Host, Vultr VPS)Linux 5.8.0-38-generic x86_64
testssl.sh 3.1dev from https://testssl.sh/dev/
(Image ID:fe90513119fb
, 5 days ago, digest:sha256:fca6197abd505668a1f5d6557c9652cd83edabe98f97a672c837cb9f9b1ca376
)/home/testssl/bin/openssl.Linux.x86_64
("OpenSSL 1.0.2-chacha (1.0.2k-dev)" [~183 ciphers] from Jan 18 17:12:17 2019
?)Additional context
When I run a single test, assertion correctly fails when the file exists:
drwetter/testssl.sh:3.1dev --jsonfile-pretty port_587.json --starttls smtp mail_ssl_cipherlists:587
I am aware of the
--overwrite
option, and the actual test suite correctly removes the files. This was just an error I noticed while writing my tests that I figured I should report. This applies to any error as far as I know, eg an invalid hostname:The docker option
--user "$(id -u):$(id -g)"
was required, to match the ownership of the directory(root "0:0") to write to IIRC,testssl.sh
was writing as1000:1000
and failing to write files due to permission errors. That's more of a docker issue and depends on context rather thantestssl.sh
though.The text was updated successfully, but these errors were encountered: