Skip to content

DISPATCH-975: Enforce max message size on message ingress v4#712

Closed
ChugR wants to merge 1 commit intoapache:masterfrom
ChugR:DISPATCH-975-40
Closed

DISPATCH-975: Enforce max message size on message ingress v4#712
ChugR wants to merge 1 commit intoapache:masterfrom
ChugR:DISPATCH-975-40

Conversation

@ChugR
Copy link
Copy Markdown
Contributor

@ChugR ChugR commented Mar 31, 2020

This update incorporates review comments and a number of new features:

* Oversize messages are enforced by closing the sender's connection

* All pn_delivery work is moved out of message.c and into router_node.c

* Oversize message handling in router_node.c is separated into an
isolated section of code making it easier to reason about how
oversize messages are handled.

* Self tests are improved

** Oversize message tests are divided into two source files; one
for basic blocking and another for more complex cases

** Each router has a different maxMessagesize

** Sixteen tests connect to each of four routers with senders and
receivers.

** Oversize and Undersize messages are checked to prove that routers
with smaller limits will successfully pass through larger messages.

** Interior routers with smaller limits than attached edge routers
will block messages that the edge routers allow but are over the
interior router limit.

TODO:

This version should not leak resources for the basic tests (system_tests_policy_oversize_basic).
This version still leaks resource (iterator, buffer, parsed_field,
message, message_content, and delivery) objects during system_tests_policy_oversize_compound tests where a message is blocked by *both* an edge router and the attached interior router.
No tests yet for multicast or link route.

====
Comments from previous pull requests

This commit has an updated implementation for review (v2 PR#698)

MaxMessageSize may be specified globally, per vhost, or per vhost user
group. The global setting applies to all vhosts for which maxMessageSize
is unspecified. The vhost setting applies to all vhost user groups for
which maxMessageSize is unspecified. The vhost user group setting
overrides all other settings. A maxMessageSize setting of zero disables
maxMessageSize enforcement.

Links over which maxMessageSize is being enforced will advertise the
size in the max-message-size field of the Attach
frame. Qpid-dispatch ignores the max-message-size field received in
incoming Attach frames.

Message size for maxMessageSize purposes is calculated to be the
number of AMQP octets in the Annotated Message. This includes the
header, delivery-annotations, message-annotations, properties,
application-properties, application-data, and footer
sections. Administrators and users must be aware that a "message"
consisting a single character string (the application-data) will be
much larger over the wire after properties and annotations have been
inserted.

Max message size is enforced on message/transfer ingress only. Once a
message has entered the router network it is free to go to any
destination.

When a message exceeds max size then:

Disposition of rejected is returned to the sender for that delivery.
Copies of the message being delivered through the router network are aborted.
Previous versions of this patch closed the sender's ingress link
with an error. This ensured that a sender would know that the
message did not get through and would clearly know why. However,
spontaneous link closures cause a wide variety of serious problems
for many clients. Link closure should not be enforced until, at a
minimum, qpid-proton clients handle the closure cleanly.

Self test includes a four-router linear network with two interior and two
edge routers. Tests try oversize and undersize messages with a variety of sender and
receiver attachment points in that network.

This update incorporates review comments and a number of new features:

	* Oversize messages are enforced by closing the sender's connection

    * All pn_delivery work is moved out of message.c and into router_node.c
    log_link_message identifies oversize messages

	* Oversize message handling in router_node.c is separated into an
    isolated section of code making it easier to reason about how
    oversize messages are handled.

    * Self tests are improved

	** Oversize message tests are divided into two source files; one
	for basic blocking and another for more complex cases

	** Each router has a different maxMessagesize

	** Sixteen tests connect to each of four routers with senders and
    receivers.

	** Oversize and Undersize messages are checked to prove that routers
    with smaller limits will successfully pass through larger messages.

	** Interior routers with smaller limits than attached edge routers
    will block messages that the edge routers allow but are over the
    interior router limit.

TODO:

    This version still leaks resource (iterator, buffer, parsed_field,
    message, message_content, and delivery) objects under some conditions.
    No tests yet for multicast or link route.

====
Comments from previous pull requests

This commit has an updated implementation for review (v2 PR#698)

MaxMessageSize may be specified globally, per vhost, or per vhost user
group. The global setting applies to all vhosts for which maxMessageSize
is unspecified. The vhost setting applies to all vhost user groups for
which maxMessageSize is unspecified. The vhost user group setting
overrides all other settings. A maxMessageSize setting of zero disables
maxMessageSize enforcement.

Links over which maxMessageSize is being enforced will advertise the
size in the max-message-size field of the Attach
frame. Qpid-dispatch ignores the max-message-size field received in
incoming Attach frames.

Message size for maxMessageSize purposes is calculated to be the
number of AMQP octets in the Annotated Message. This includes the
header, delivery-annotations, message-annotations, properties,
application-properties, application-data, and footer
sections. Administrators and users must be aware that a "message"
consisting a single character string (the application-data) will be
much larger over the wire after properties and annotations have been
inserted.

Max message size is enforced on message/transfer ingress only. Once a
message has entered the router network it is free to go to any
destination.

When a message exceeds max size then:

    Disposition of rejected is returned to the sender for that delivery.
    Copies of the message being delivered through the router network are aborted.
    Previous versions of this patch closed the sender's ingress link
    with an error. This ensured that a sender would know that the
    message did not get through and would clearly know why. However,
    spontaneous link closures cause a wide variety of serious problems
    for many clients. Link closure should not be enforced until, at a
    minimum, qpid-proton clients handle the closure cleanly.

Self test includes a four-router linear network with two interior and two
edge routers. Tests try oversize and undersize messages with a variety of sender and
receiver attachment points in that network.
@ChugR
Copy link
Copy Markdown
Contributor Author

ChugR commented Apr 8, 2020

New PR on the way

@ChugR ChugR closed this Apr 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant