Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: simplify Networkwith Framed<.., Codec> #825

Merged
merged 9 commits into from
Mar 25, 2024
Merged

refactor: simplify Networkwith Framed<.., Codec> #825

merged 9 commits into from
Mar 25, 2024

Conversation

de-sh
Copy link
Member

@de-sh de-sh commented Mar 19, 2024

As discussed in #814 this simplifies Network using Framed and Codec

Type of change

  • mqttbytes::Error is no longer Clone, Copy, PartialEq, Eq and 2 new variants, for io and packet size limit
  • Refactors different parts of the codebase to regularize and simplify.
  • Deprecates bulk read and moves write buffer into Network, which might lead to issues with packets in the buffer being dropped when an error is faced. But this seems like premature-optimization to be frank

Checklist:

  • Formatted with cargo fmt
  • Make an entry to CHANGELOG.md if it's relevant to the users of the library. If it's not relevant mention why.

@coveralls
Copy link

coveralls commented Mar 19, 2024

Pull Request Test Coverage Report for Build 8346655192

Details

  • 274 of 519 (52.79%) changed or added relevant lines in 18 files are covered.
  • 74 unchanged lines in 7 files lost coverage.
  • Overall coverage decreased (-0.3%) to 37.53%

Changes Missing Coverage Covered Lines Changed/Added Lines %
rumqttc/src/mqttbytes/mod.rs 0 1 0.0%
rumqttc/src/v5/mqttbytes/mod.rs 0 1 0.0%
rumqttc/src/framed.rs 19 21 90.48%
rumqttc/src/mqttbytes/v4/codec.rs 29 31 93.55%
rumqttc/src/proxy.rs 0 3 0.0%
rumqttc/src/tls.rs 0 3 0.0%
rumqttc/src/mqttbytes/v4/connack.rs 0 6 0.0%
rumqttc/src/v5/mqttbytes/v5/connect.rs 0 6 0.0%
rumqttc/src/v5/mqttbytes/v5/ping.rs 0 6 0.0%
rumqttc/src/v5/mqttbytes/v5/codec.rs 22 31 70.97%
Files with Coverage Reduction New Missed Lines %
rumqttc/src/lib.rs 1 64.32%
rumqttc/src/mqttbytes/mod.rs 3 86.27%
rumqttc/src/v5/state.rs 5 68.42%
rumqttc/src/state.rs 11 79.36%
rumqttc/src/mqttbytes/v4/pubcomp.rs 18 8.33%
rumqttc/src/mqttbytes/v4/pubrel.rs 18 8.11%
rumqttc/src/mqttbytes/v4/pubrec.rs 18 8.11%
Totals Coverage Status
Change from base Build 8243857492: -0.3%
Covered Lines: 6239
Relevant Lines: 16624

💛 - Coveralls

Copy link
Contributor

@flxo flxo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I appreciate the introduction of Framed!

The PR is definitely a step into right direction but I completely miss the batching topic discussed in #810.

Furthermore this patch removes the previously implemented batch sending of packets that are generated in response to incoming frames. The current implementation queues up to 10 packets before calling flush. This PR flushes the response after every processed incoming frame. I haven't profiled but this definitely has a negative performance impact.

Calling Network::send results in a SinkExt::send (impled by Framed). This is basically a poll_ready + start_send + poll_flush. This results in a sendto or similar sys call per packet.

To get back proper batching of outgoing frames there should made use of SinkExt::feed in a limited loop + SinkExt::flush. Think I demonstrated this here.

Is there a follow up to this PR already planned?

@de-sh
Copy link
Member Author

de-sh commented Mar 20, 2024

Is there a follow up to this PR already planned?

Thanks for the suggestions, have opened #826 to resolve this, please review!

@de-sh
Copy link
Member Author

de-sh commented Mar 25, 2024

Merging this, no functional changes included in this PR

* ack incoming publishes if required
* remove unused write buffer
* update tests
@de-sh de-sh merged commit f869eae into main Mar 25, 2024
1 of 4 checks passed
@de-sh de-sh deleted the codec branch March 25, 2024 10:02
@xiaocq2001
Copy link
Contributor

After it's merged, some incoming messages disappear in event loop, please help to check.

asyncpubsub_v5 output before the merge:

     Running `target/debug/examples/asyncpubsub_v5`
Event = Incoming(ConnAck(ConnAck { session_present: false, code: Success, properties: Some(ConnAckProperties { session_expiry_interval: None, receive_max: Some(20), max_qos: None, retain_available: None, max_packet_size: None, assigned_client_identifier: None, topic_alias_max: Some(10), reason_string: None, user_properties: [], wildcard_subscription_available: None, subscription_identifiers_available: None, shared_subscription_available: None, server_keep_alive: None, response_information: None, server_reference: None, authentication_method: None, authentication_data: None }) }))
Event = Outgoing(Subscribe(1))
Event = Outgoing(Publish(2))
Event = Incoming(SubAck(SubAck { pkid: 1, return_codes: [Success(AtMostOnce)], properties: None }))
Event = Outgoing(PubRel(2))
Event = Incoming(PubRec(PubRec { pkid: 2, reason: Success, properties: None }))
Event = Incoming(Publish(Publish { dup: false, qos: AtMostOnce, retain: false, topic: b"hello/world", pkid: 0, payload: b"\x01", properties: None }))

asyncpubsub_v5 output after merge:

     Running `target/debug/examples/asyncpubsub_v5`
Event = Incoming(ConnAck(ConnAck { session_present: false, code: Success, properties: Some(ConnAckProperties { session_expiry_interval: None, receive_max: Some(20), max_qos: None, retain_available: None, max_packet_size: None, assigned_client_identifier: None, topic_alias_max: Some(10), reason_string: None, user_properties: [], wildcard_subscription_available: None, subscription_identifiers_available: None, shared_subscription_available: None, server_keep_alive: None, response_information: None, server_reference: None, authentication_method: None, authentication_data: None }) }))
Event = Outgoing(Subscribe(1))
Event = Outgoing(Publish(2))
Event = Outgoing(Publish(3))
Event = Outgoing(Publish(4))

de-sh added a commit that referenced this pull request Apr 7, 2024
* fix: v5 doesn't write outgoing packets onto network

#825 (comment)

* same for pingreq
@de-sh
Copy link
Member Author

de-sh commented Apr 7, 2024

Thanks for pointing this out @xiaocq2001 there is the lack of a good testsuite for v5, might be a good issue to open!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants