Handling too long comment

[hiro-o918]

Overview

github-comment currently manages excessively long comments by silently replacing them with the BodyForTooLong variable. This approach can cause confusion, especially when an empty comment is posted.

I've also noticed that the maximum comment length set within github-comment differs from the limit established by the GitHub API.
The comments treated as too long by github-comment can still be posted via GitHub's GUI.

So, I have several proposals

1. Add warning log when replacing the body with BodyForTooLong
2. Replace the body when GitHub API responses corresponding error.

In relation to the second proposal, I've found this pull request: 1bbb5e1, which made changes to use the hard coded comment length limit instead of relying on the API's error handling.
Therefore, I'd like to discuss which approach is the most effective for managing comments that exceed the permissible length.

Thank you for providing such a useful tool!

Why is the proposal needed?

By this proposal, developer can debugging github-comment more easily

Reference

[suzuki-shunsuke]

Thank you for your proposal.

github-comment has two features for too long comments.

1. BodyForTooLong
2. Omit the template hidden_combined_output if hidden_combined_output is too long https://github.com/suzuki-shunsuke/github-comment/releases/tag/v5.1.0

I'm sorry but maybe the option 2 is undocumented. I'll check and update the document.
I think the option 2 is enough for almost use cases.

Add warning log when replacing the body with BodyForTooLong

Yeah, this is a good suggestion. Let's add the warning.

Replace the body when GitHub API responses corresponding error.

This is the feature of BodyForTooLong, right?

I've also noticed that the maximum comment length set within github-comment differs from the limit established by the GitHub API.
The comments treated as too long by github-comment can still be posted via GitHub's GUI.

Maybe the condition len(cmt.Body) > 65536 is wrong? 🤔

github-comment/pkg/github/comment.go

Line 83 in 1516148

return client.createComment(ctx, cmt, len(cmt.Body) > 65536) //nolint:gomnd

[hiro-o918]

Yes, this suggestion is for the feature BodyForTooLong.
I didn't check len(cmt.Body) > 65536 condition precisely, but it seems to be wrong.

So another option is update the 65536 to be correct.
But it is hard to watch the specifications of API and keep the value to be updated.

My suggestion is to use the following part, which handling errors of GitHub API.

github-comment/pkg/comment/comment.go

Lines 110 to 121 in 86550fe

	e := &httpclient.Error{}
	if errors.As(err, &e) {
	validationErrors := ValidationErrors{}
	if err := json.Unmarshal(e.BodyByte(), &validationErrors); err == nil {
	for _, ve := range validationErrors.Errors {
	if strings.HasPrefix(ve.Message, "Body is too long") {
	return commenter.create(ctx, cmt, true)
	}
	}
	}
	}
	return err

As you know, there are some pros and cons between updating the value and handling API error.

pros of handling error

• we do not need to update the limit value to be updated
  • probably, valid limitation is not published in official
    • error: body is too long (maximum is 65536 characters) mshick/add-pr-comment#93 (comment)
• larger comment can be posted because it follows the limit of GitHub API

cons of handling error

• If the error body is changed, all the old versions does not work well.

Which approach appears to be more effective?

[suzuki-shunsuke]

You referred to the package pkg/comment, but this package was removed at v4.5.0.

• feat: get an associated pr with commit hash #545

At v4.5.0, how to handle too long comment was changed.

github-comment v4.5.0 and below posted a comment with a normal template first, and if it failed to post a comment github-comment posted a comment with BodyForTooLong.
I think this is what you suggested.

On the other hand, github-comment v4.5.0 and above checks the length of a normal template first, and if the comment is longer than threshold 65536, github-comment posts a comment with BodyForTooLong.

Which approach appears to be more effective?

The advantage of the latter is to decrease the useless API call.
But if the validation is incorrect, I think the former approach is better.

But as I said github-comment v5.1.0 and above omits the template hidden_combined_output if hidden_combined_output is too long. I think this feature omits almost too long comment.

[hiro-o918]

Sorry for late reply.

github-comment v4.5.0 and below posted a comment with a normal template first, and if it failed to post a comment github-comment posted a comment with BodyForTooLong.
I think this is what you suggested.

Yes, I suggested the deleted feature.

I understand, basically the length of comments does not exceed the limit because hidden_combined_output is trimmed from the body.

If I meet more cases the body is too long, let me create a PR to fallback to BodyForTooLong.

Thank you for your answering.