Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement fragmentation of large distribution messages #2133

Merged
merged 13 commits into from
Feb 22, 2019
Merged

Implement fragmentation of large distribution messages #2133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
bcb79fa
erts: Limit binary printout for %.XT in erts_print
garazdawi Jan 9, 2019
5c8f2be
erts: Refactor rbt _yielding to use reductions
garazdawi Dec 17, 2018
45c5725
erts: Yield later during process exit and allow free procs to run
garazdawi Dec 17, 2018
0184c2e
erts: Remove a copy of distribution data payload
garazdawi Sep 18, 2018
9a7df41
erts: Move reason in dist messages to payload
garazdawi Sep 19, 2018
6686877
erts: Expand distribution protocol documentation
garazdawi Sep 26, 2018
f2c4f6f
erts: Implement fragmentation of distrubution messages
garazdawi Sep 26, 2018
d191345
erts: Make remote send of exit/2 trap
garazdawi Jan 25, 2019
6493f5e
erts: Add distr testcases for fragmentation
garazdawi Jan 25, 2019
1066040
erts: Refactor ErtsSendContext to be ErtsDSigSendContext
garazdawi Jan 29, 2019
2bf27ec
erts: Add ERL_NODE_BOOKKEEP to node tables refc
garazdawi Jan 31, 2019
fc09673
erts: Implement trapping while sending distr exit/down
garazdawi Jan 31, 2019
c0c6f6b
erts: Expand etp to look for free processes
garazdawi Jan 31, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 101 additions & 15 deletions erts/doc/src/erl_dist_protocol.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -532,11 +532,7 @@ io:format("old/unused name ~ts at port ~p, fd = ~p ~n",
<marker id="distribution_handshake"/>
<title>Distribution Handshake</title>
<p>This section describes the distribution handshake protocol introduced
in Erlang/OTP R6. This description was previously located in
<c>$ERL_TOP/lib/kernel/internal_doc/distribution_handshake.txt</c> and
has more or less been copied and "formatted" here. It has been almost
unchanged since 1999, but the handshake has not changed much since then
either.</p>
in Erlang/OTP R6. The handshake has remained almost the same since then.</p>

<section>
<title>General</title>
Expand Down Expand Up @@ -847,6 +843,18 @@ DiB == gen_digest(ChA, ICA)?
of the <c>SEND_TT</c> control message.
</p>
</item>
<tag><c>-define(DFLAG_BIG_SEQTRACE_LABELS, 16#100000).</c></tag>
<item>
<p>The node understands any term as the seqtrace label.</p>
</item>
<tag><c>-define(DFLAG_EXIT_PAYLOAD, 16#400000).</c></tag>
<item>
<p>Use the <c>PAYLOAD_EXIT</c>, <c>PAYLOAD_EXIT_TT</c>,
<c>PAYLOAD_EXIT2</c>, <c>PAYLOAD_EXIT2_TT</c>
and <c>PAYLOAD_MONITOR_P_EXIT</c>
<seealso marker="#control_message">control message</seealso>s
instead of the non-PAYLOAD variants.</p>
</item>
</taglist>
<p>
There is also function <c>dist_util:strict_order_flags/0</c>
Expand All @@ -859,7 +867,7 @@ DiB == gen_digest(ChA, ICA)?
<section>
<marker id="connected_nodes"/>
<title>Protocol between Connected Nodes</title>
<p>As from ERTS 5.7.2 the runtime system passes a distribution flag
<p>Since ERTS 5.7.2 (OTP R13B) the runtime system passes a distribution flag
in the handshake stage that enables the use of a
<seealso marker="erl_ext_dist#distribution_header">distribution header
</seealso> on all messages passed. Messages passed between nodes have in
Expand All @@ -878,7 +886,7 @@ DiB == gen_digest(ChA, ICA)?
<cell align="center"><c>ControlMessage</c></cell>
<cell align="center"><c>Message</c></cell>
</row>
<tcaption>Format of Messages Passed between Nodes (as from ERTS 5.7.2)
<tcaption>Format of Messages Passed between Nodes (as from ERTS 5.7.2 (OTP R13B))
</tcaption>
</table>

Expand All @@ -887,23 +895,31 @@ DiB == gen_digest(ChA, ICA)?
<item>
<p>Equal to d + n + m.</p>
</item>
<tag><c>DistributionHeader</c></tag>
<item>
<p>
<seealso marker="erl_ext_dist#distribution_header">Distribution header
describing the atom cache and fragmented distribution messages.
</seealso>
</p>
</item>
<tag><c>ControlMessage</c></tag>
<item>
<p>A tuple passed using the external format of Erlang.</p>
</item>
<tag><c>Message</c></tag>
<item>
<p>The message sent to another node using the '!' (in external format).
Notice that <c>Message</c> is only passed in combination with a
<c>ControlMessage</c> encoding a send ('!').</p>
<p>The message sent to another node using the '!'
or the reason for a EXIT, EXIT2 or DOWN signal using
the external term format.</p>
</item>
</taglist>

<p>Notice that <seealso marker="erl_ext_dist#overall_format">the version
number is omitted from the terms that follow a distribution header
</seealso>.</p>

<p>Nodes with an ERTS version earlier than 5.7.2 does not pass the
<p>Nodes with an ERTS version earlier than 5.7.2 (OTP R13B) does not pass the
distribution flag that enables the distribution header. Messages passed
between nodes have in this case the following format:</p>

Expand All @@ -920,7 +936,7 @@ DiB == gen_digest(ChA, ICA)?
<cell align="center"><c>ControlMessage</c></cell>
<cell align="center"><c>Message</c></cell>
</row>
<tcaption>Format of Messages Passed between Nodes (before ERTS 5.7.2)
<tcaption>Format of Messages Passed between Nodes (before ERTS 5.7.2 (OTP R13B))
</tcaption>
</table>

Expand Down Expand Up @@ -963,6 +979,7 @@ DiB == gen_digest(ChA, ICA)?
<tag><c>EXIT</c></tag>
<item>
<p><c>{3, FromPid, ToPid, Reason}</c></p>
<p>This signal is sent when a link has been broken</p>
</item>
<tag><c>UNLINK</c></tag>
<item>
Expand All @@ -985,6 +1002,7 @@ DiB == gen_digest(ChA, ICA)?
<tag><c>EXIT2</c></tag>
<item>
<p><c>{8, FromPid, ToPid, Reason}</c></p>
<p>This signal is sent by a call to the erlang:exit/2 bif</p>
</item>
</taglist>
</section>
Expand All @@ -1007,7 +1025,7 @@ DiB == gen_digest(ChA, ICA)?
<p><c>{16, FromPid, Unused, ToName, TraceToken}</c></p>
<p>Followed by <c>Message</c>.</p>
<p><c>Unused</c> is kept for backward compatibility.</p>
</item>
</item>
<tag><c>EXIT2_TT</c></tag>
<item>
<p><c>{18, FromPid, ToPid, TraceToken, Reason}</c></p>
Expand Down Expand Up @@ -1061,7 +1079,7 @@ DiB == gen_digest(ChA, ICA)?
<p><c>{22, FromPid, ToPid}</c></p>
<p>Followed by <c>Message</c>.</p>
<p>
This control messages replace the <c>SEND</c> control
This control message replaces the <c>SEND</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
has been negotiated in the connection setup handshake.
Expand All @@ -1080,7 +1098,7 @@ DiB == gen_digest(ChA, ICA)?
<p><c>{23, FromPid, ToPid, TraceToken}</c></p>
<p>Followed by <c>Message</c>.</p>
<p>
This control messages replace the <c>SEND_TT</c> control
This control message replaces the <c>SEND_TT</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_SEND_SENDER</c></seealso>
has been negotiated in the connection setup handshake.
Expand All @@ -1097,4 +1115,72 @@ DiB == gen_digest(ChA, ICA)?
</taglist>
</section>

<section>
<title>New Ctrlmessages for Erlang/OTP 22</title>
<note><p>
Messages encoded before the connection has
been set up may still use the non-PAYLOAD variant.
However, once a PAYLOAD control message has been sent,
no more non-PAYLOAD control messages will be sent in
the same direction on the connection.
</p></note>
<taglist>
<tag><c>PAYLOAD_EXIT</c></tag>
<item>
<p><c>{24, FromPid, ToPid}</c></p>
<p>Followed by <c>Reason</c>.</p>
<p>
This control message replaces the <c>EXIT</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_EXIT_PAYLOAD</c></seealso>
has been negotiated in the connection setup handshake.
</p>
</item>
<tag><c>PAYLOAD_EXIT_TT</c></tag>
<item>
<p><c>{25, FromPid, ToPid}</c></p>
<p>Followed by <c>Reason</c>.</p>
<p>
This control message replaces the <c>EXIT_TT</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_EXIT_PAYLOAD</c></seealso>
has been negotiated in the connection setup handshake.
</p>
</item>
<tag><c>PAYLOAD_EXIT2</c></tag>
<item>
<p><c>{26, FromPid, ToPid}</c></p>
<p>Followed by <c>Reason</c>.</p>
<p>
This control message replaces the <c>EXIT2</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_EXIT_PAYLOAD</c></seealso>
has been negotiated in the connection setup handshake.
</p>
</item>
<tag><c>PAYLOAD_EXIT2_TT</c></tag>
<item>
<p><c>{27, FromPid, ToPid}</c></p>
<p>Followed by <c>Reason</c>.</p>
<p>
This control message replaces the <c>EXIT2_TT</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_EXIT_PAYLOAD</c></seealso>
has been negotiated in the connection setup handshake.
</p>
</item>
<tag><c>PAYLOAD_MONITOR_P_EXIT</c></tag>
<item>
<p><c>{28, FromPid, ToPid, Ref}</c></p>
<p>Followed by <c>Reason</c>.</p>
<p>
This control message replaces the <c>MONITOR_P_EXIT</c> control
message and will be sent when the distribution flag
<seealso marker="erl_dist_protocol#dflags"><c>DFLAG_EXIT_PAYLOAD</c></seealso>
has been negotiated in the connection setup handshake.
</p>
</item>
</taglist>
</section>

</chapter>
9 changes: 1 addition & 8 deletions erts/doc/src/erl_ext_dist.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,15 +136,8 @@
</note>
</section>

<section>
<section>
<title>Distribution Header</title>
<p>
<marker id="distribution_header"/>
As from ERTS 5.7.2 the old atom cache protocol was
dropped and a new one was introduced. This protocol
introduced the distribution header. Nodes with an ERTS version
earlier than 5.7.2 can still communicate with new nodes,
but no distribution header and no atom cache are used.</p>
<p>
The distribution header only contains an atom cache
reference section, but can in the future contain more
Expand Down
5 changes: 4 additions & 1 deletion erts/emulator/beam/beam_emu.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -885,19 +885,22 @@ void process_main(Eterm * x_reg_array, FloatDef* f_reg_array)
#include "beam_warm.h"

OpCase(normal_exit): {
SWAPOUT;
HEAVY_SWAPOUT;
c_p->freason = EXC_NORMAL;
c_p->arity = 0; /* In case this process will ever be garbed again. */
ERTS_UNREQ_PROC_MAIN_LOCK(c_p);
erts_do_exit_process(c_p, am_normal);
ERTS_REQ_PROC_MAIN_LOCK(c_p);
HEAVY_SWAPIN;
goto do_schedule;
}

OpCase(continue_exit): {
HEAVY_SWAPOUT;
ERTS_UNREQ_PROC_MAIN_LOCK(c_p);
erts_continue_exit_process(c_p);
ERTS_REQ_PROC_MAIN_LOCK(c_p);
HEAVY_SWAPIN;
goto do_schedule;
}

Expand Down
3 changes: 2 additions & 1 deletion erts/emulator/beam/beam_load.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6108,7 +6108,8 @@ erts_release_literal_area(ErtsLiteralArea* literal_area)
}
default:
ASSERT(is_external_header(oh->thing_word));
erts_deref_node_entry(((ExternalThing*)oh)->node);
erts_deref_node_entry(((ExternalThing*)oh)->node,
make_boxed(&oh->thing_word));
}
oh = oh->next;
}
Expand Down
Loading