Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: don't be kill when send large payload when log is debug #11279

Merged
merged 2 commits into from Aug 10, 2023

Conversation

zhongwencool
Copy link
Member

@zhongwencool zhongwencool commented Jul 17, 2023

Fixes https://emqx.atlassian.net/browse/EMQX-10482

Summary

馃 Generated by Copilot at b277d4a

This pull request improves the performance and readability of the emqx_packet and emqx_trace_formatter modules by using OTP 24 features and refactoring some code. It also fixes some test settings and limits the output of large payloads in the trace and packet formatting functions.

PR Checklist

Please convert it to a draft if any of the following conditions are not met. Reviewers may skip over until all the items are checked:

  • Added tests for the changes
  • Changed lines covered in coverage report
  • Change log has been added to changes/(ce|ee)/(feat|perf|fix)-<PR-id>.en.md files
  • For internal contributor: there is a jira ticket to track this change
  • If there should be document changes, a PR to emqx-docs.git is sent, or a jira ticket is created to follow up
  • Schema changes are backward compatible

Checklist for CI (.github/workflows) changes

  • If changed package build workflow, pass this action (manual trigger)
  • Change log has been added to changes/ dir for user-facing artifacts update

@zhongwencool zhongwencool requested review from a team, lafirest and sstrigler as code owners July 17, 2023 14:22
format_payload(Payload, hex) when ?MAX_PAYLOAD_FORMAT_LIMIT(Payload) ->
["Payload(hex)=", binary:encode_hex(Payload)];
format_payload(_, hidden) ->
"Payload=******";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or only display a part to keep consistent with the above

format_payload(_, hidden) ->
"Payload=******";
format_payload(Payload, _) ->
["Payload=Large data detected (", integer_to_list(byte_size(Payload)), " bytes)"].
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @lafirest, perhaps it's better to display ?MAX_PAYLOAD_FORMAT_SIZE characters in this case

@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 5815911358

  • 2 of 9 (22.22%) changed or added relevant lines in 2 files are covered.
  • 34 unchanged lines in 13 files lost coverage.
  • Overall coverage increased (+0.03%) to 81.439%

Changes Missing Coverage Covered Lines Changed/Added Lines %
apps/emqx/src/emqx_packet.erl 1 4 25.0%
apps/emqx/src/emqx_trace/emqx_trace_formatter.erl 1 5 20.0%
Files with Coverage Reduction New Missed Lines %
apps/emqx_bridge_gcp_pubsub/src/emqx_bridge_gcp_pubsub_client.erl 1 94.74%
apps/emqx_bridge_gcp_pubsub/src/emqx_bridge_gcp_pubsub_consumer_worker.erl 1 93.31%
apps/emqx_conf/src/emqx_cluster_rpc.erl 1 86.86%
apps/emqx_ft/src/emqx_ft_assembly.erl 1 97.12%
apps/emqx_gateway_exproto/src/emqx_exproto_gcli.erl 1 64.15%
apps/emqx_gateway/src/bhvrs/emqx_gateway_conn.erl 1 75.88%
apps/emqx/src/emqx_broker.erl 1 88.33%
apps/emqx/src/emqx_connection.erl 1 83.47%
apps/emqx_conf/src/emqx_conf_app.erl 2 84.91%
apps/emqx_ft/src/emqx_ft_storage_exporter_fs.erl 3 78.26%
Totals Coverage Status
Change from base Build 5813313118: 0.03%
Covered Lines: 31972
Relevant Lines: 39259

馃挍 - Coveralls

@zhongwencool zhongwencool merged commit d1dc37a into emqx:master Aug 10, 2023
125 of 126 checks passed
@zhongwencool zhongwencool deleted the trace-log branch August 10, 2023 03:59
@@ -679,4 +679,7 @@ end).
-define(THROW_FRAME_ERROR(Reason), erlang:throw({?FRAME_PARSE_ERROR, Reason})).
-define(THROW_SERIALIZE_ERROR(Reason), erlang:throw({?FRAME_SERIALIZE_ERROR, Reason})).

-define(MAX_PAYLOAD_FORMAT_SIZE, 1024).
-define(MAX_PAYLOAD_FORMAT_LIMIT(Bin), (byte_size(Bin) =< ?MAX_PAYLOAD_FORMAT_SIZE)).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: UNDER_MAX....

format_payload(Payload, hex) when ?MAX_PAYLOAD_FORMAT_LIMIT(Payload) -> binary:encode_hex(Payload);
format_payload(<<Part:100, _/binary>> = Payload, _) ->
[
Part,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format Part according to 2nd arg.

format_payload(<<Part:100, _/binary>> = Payload, _) ->
[
"Payload=",
Part,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

format Part according to 2nd arg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants