Skip to content

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
...
Checking mergeability… Don’t worry, you can still create the pull request.
  • 10 commits
  • 10 files changed
  • 0 commit comments
  • 4 contributors
Commits on Feb 03, 2016
@sirihansen sirihansen Add supervisor:get_callback_module/1
This function is used by release_handler during upgrade. This was
earlier implemented in the release_handler, but it required a copy og
the definition of the supervisor's internal state, which caused
problems when this state was updated.
4422cb3
@nybek nybek Speed up supervisor:count_children/1; simple_one_for_one
Speed up supervisor:count_children/1 for simple_one_for_one
supervisors. This is achieved by avoiding looping through all the
child process and verifying that each one is alive.

For a supervisor with 100,000 'temporary' children the count-time will
drop from approx 25ms to about 0.005ms.

For a supervisor with 100,000 'permanent' or 'transient' children the
count-time will drop from approx 30ms to about 0.005ms.

This avoids having the supervisor block for an extended period while
the count takes place. Under normal circumstances the accuracy of the
result should also improve since the duration is too short for many
processes to die during the count.
264a5e8
Commits on Feb 08, 2016
@bjorng bjorng Eliminate crash because of unsafe delaying of sub-binary creation
The following code would fail to compile:

decode(<<Code/integer, Bin/binary>>) ->
    <<C1/integer, B1/binary>> = Bin,
    case C1 of
	X when X =:= 1 orelse X =:= 2 ->
	    Bin2 = <<>>;
	_ ->
	    Bin2 = B1
    end,
    case Code of
	1 -> decode(Bin2);
	_ -> Bin2
    end.

The error message would be:

t: function decode/1+28:
  Internal consistency check failed - please report this bug.
  Instruction: return
  Error:       {match_context,{x,0}}:

The beam_bsm pass would delay the creation of a sub-binary when it was
unsafe to do so. The culprit was the btb_follow_branch/3 function that
for performance reasons cached labels that had already been checked.
The problem was the safety of a label also depends on the contents
of the registers. Therefore, the key for caching needs to be both
the label and the register contents.

Reported-by: José Valim
abde1d0
Commits on Feb 10, 2016
@sirihansen sirihansen Merge branch 'nybek/speed_up_supervisor_count_children' into maint
* nybek/speed_up_supervisor_count_children:
  Speed up supervisor:count_children/1; simple_one_for_one
  Add supervisor:get_callback_module/1

OTP-13290
8fba155
@sirihansen sirihansen Merge branch 'maint'
Conflicts:
	lib/stdlib/src/supervisor.erl
505a697
@bjorng bjorng Merge branch 'bjorn/compiler/beam_bsm/OTP-13309' into maint
* bjorn/compiler/beam_bsm/OTP-13309:
  Eliminate crash because of unsafe delaying of sub-binary creation
47c2ace
@bjorng bjorng Merge branch 'maint'
* maint:
  Eliminate crash because of unsafe delaying of sub-binary creation
3fd5886
@psyeugenic psyeugenic tools: Remove dead code in lcnt e647414
Commits on Feb 11, 2016
@psyeugenic psyeugenic Merge branch 'egil/fix-lcnt-dead-code/OTP-13319' into maint
* egil/fix-lcnt-dead-code/OTP-13319:
  tools: Remove dead code in lcnt
5c023ea
@psyeugenic psyeugenic Merge branch 'maint' d7b8fad
View
5 lib/compiler/src/beam_bsm.erl
@@ -421,7 +421,8 @@ btb_follow_branches([], _, D) -> D.
btb_follow_branch(0, _Regs, D) -> D;
btb_follow_branch(Lbl, Regs, #btb{ok_br=Br0,index=Li}=D) ->
- case gb_sets:is_member(Lbl, Br0) of
+ Key = {Lbl,Regs},
+ case gb_sets:is_member(Key, Br0) of
true ->
%% We have already followed this branch and it was OK.
D;
@@ -432,7 +433,7 @@ btb_follow_branch(Lbl, Regs, #btb{ok_br=Br0,index=Li}=D) ->
btb_reaches_match_1(Is, Regs, D),
%% Since we got back, this branch is OK.
- D#btb{ok_br=gb_sets:insert(Lbl, Br),must_not_save=MustNotSave,
+ D#btb{ok_br=gb_sets:insert(Key, Br),must_not_save=MustNotSave,
must_save=MustSave}
end.
View
30 lib/compiler/test/bs_match_SUITE.erl
@@ -37,7 +37,7 @@
cover_beam_bool/1,matched_out_size/1,follow_fail_branch/1,
no_partition/1,calling_a_binary/1,binary_in_map/1,
match_string_opt/1,select_on_integer/1,
- map_and_binary/1]).
+ map_and_binary/1,unsafe_branch_caching/1]).
-export([coverage_id/1,coverage_external_ignore/2]).
@@ -64,7 +64,7 @@ groups() ->
cover_beam_bool,matched_out_size,follow_fail_branch,
no_partition,calling_a_binary,binary_in_map,
match_string_opt,select_on_integer,
- map_and_binary]}].
+ map_and_binary,unsafe_branch_caching]}].
init_per_suite(Config) ->
@@ -1267,6 +1267,32 @@ do_map_and_binary(#{time := _} = T) ->
do_map_and_binary(#{hour := Hour, min := Min} = T) ->
{Hour, Min, T}.
+%% Unsafe caching of branch outcomes in beam_bsm would cause the
+%% delayed creation of sub-binaries optimization to be applied even
+%% when it was unsafe.
+
+unsafe_branch_caching(_Config) ->
+ <<>> = do_unsafe_branch_caching(<<42,1>>),
+ <<>> = do_unsafe_branch_caching(<<42,2>>),
+ <<>> = do_unsafe_branch_caching(<<42,3>>),
+ <<17,18>> = do_unsafe_branch_caching(<<42,3,17,18>>),
+ <<>> = do_unsafe_branch_caching(<<1,3,42,2>>),
+
+ ok.
+
+do_unsafe_branch_caching(<<Code/integer, Bin/binary>>) ->
+ <<C1/integer, B1/binary>> = Bin,
+ case C1 of
+ X when X =:= 1 orelse X =:= 2 ->
+ Bin2 = <<>>;
+ _ ->
+ Bin2 = B1
+ end,
+ case Code of
+ 1 -> do_unsafe_branch_caching(Bin2);
+ _ -> Bin2
+ end.
+
check(F, R) ->
R = F().
View
33 lib/sasl/src/release_handler_1.erl
@@ -676,48 +676,19 @@ maybe_get_dynamic_mods(Name, Pid) ->
error(get_modules_failed)
end.
-%% XXXX
-%% Note: The following is a terrible hack done in order to resolve the
-%% problem stated in ticket OTP-3452.
-
-%% XXXX NOTE WELL: This record is from supervisor.erl. Also the record
-%% name is really `state'.
--record(supervisor_state, {name,
- strategy,
- children = [],
- dynamics = [],
- intensity,
- period,
- restarts = [],
- module,
- args}).
-
%% Return the name of the call-back module that implements the
%% (top) supervisor SupPid.
%% Returns: {ok, Module} | {error,undefined}
%%
get_supervisor_module(SupPid) ->
- case catch get_supervisor_module1(SupPid) of
- {ok, Module} when is_atom(Module) ->
+ case catch supervisor:get_callback_module(SupPid) of
+ Module when is_atom(Module) ->
{ok, Module};
_Other ->
io:format("~w: reason: ~w~n", [SupPid, _Other]),
{error, undefined}
end.
-get_supervisor_module1(SupPid) ->
- {status, _Pid, {module, _Mod},
- [_PDict, _SysState, _Parent, _Dbg, Misc]} = sys:get_status(SupPid),
- %% supervisor Misc field changed at R13B04, handle old and new variants here
- State = case Misc of
- [_Name, State1, _Type, _Time] ->
- State1;
- [_Header, _Data, {data, [{"State", State2}]}] ->
- State2
- end,
- %% Cannot use #supervisor_state{module = Module} = State.
- {ok, element(#supervisor_state.module, State)}.
-
%%-----------------------------------------------------------------
%% Func: do_soft_purge/3
%% Args: Mod = atom()
View
2 lib/sasl/src/sasl.app.src
@@ -46,6 +46,6 @@
{env, [{sasl_error_logger, tty},
{errlog_type, all}]},
{mod, {sasl, []}},
- {runtime_dependencies, ["tools-2.6.14","stdlib-2.6","kernel-4.1",
+ {runtime_dependencies, ["tools-2.6.14","stdlib-2.8","kernel-4.1",
"erts-6.0"]}]}.
View
2 lib/sasl/test/release_handler_SUITE.erl
@@ -1363,7 +1363,7 @@ upgrade_supervisor(Conf) when is_list(Conf) ->
%% Check that the restart strategy and child spec is updated
{status, _, {module, _}, [_, _, _, _, [_,_,{data,[{"State",State}]}]]} =
rpc:call(Node,sys,get_status,[a_sup]),
- {state,_,RestartStrategy,[Child],_,_,_,_,_,_} = State,
+ {state,_,RestartStrategy,[Child],_,_,_,_,_,_,_} = State,
one_for_all = RestartStrategy, % changed from one_for_one
{child,_,_,_,_,brutal_kill,_,_} = Child, % changed from timeout 2000
View
5 lib/stdlib/doc/src/supervisor.xml
@@ -543,7 +543,10 @@
</item>
<item>
<p><c>active</c> - the count of all actively running child processes
- managed by this supervisor.</p>
+ managed by this supervisor. In the case of <c>simple_one_for_one</c>
+ supervisors, no check is carried out to ensure that each child process
+ is still alive, though the result provided here is likely to be very
+ accurate unless the supervisor is heavily overloaded.</p>
</item>
<item>
<p><c>supervisors</c> - the count of all children marked as
View
63 lib/stdlib/src/supervisor.erl
@@ -33,6 +33,9 @@
terminate/2, code_change/3]).
-export([try_again_restart/2]).
+%% For release_handler only
+-export([get_callback_module/1]).
+
%%--------------------------------------------------------------------------
-export_type([sup_flags/0, child_spec/0, startchild_ret/0, strategy/0]).
@@ -115,6 +118,7 @@
intensity :: non_neg_integer() | 'undefined',
period :: pos_integer() | 'undefined',
restarts = [],
+ dynamic_restarts = 0 :: non_neg_integer(),
module,
args}).
-type state() :: #state{}.
@@ -252,6 +256,17 @@ try_again_restart(Supervisor, Child) ->
cast(Supervisor, Req) ->
gen_server:cast(Supervisor, Req).
+%%%-----------------------------------------------------------------
+%%% Called by release_handler during upgrade
+-spec get_callback_module(Pid) -> Module when
+ Pid :: pid(),
+ Module :: atom().
+get_callback_module(Pid) ->
+ {status, _Pid, {module, _Mod},
+ [_PDict, _SysState, _Parent, _Dbg, Misc]} = sys:get_status(Pid),
+ [_Header, _Data, {data, [{"State", State}]}] = Misc,
+ State#state.module.
+
%%% ---------------------------------------------------
%%%
%%% Initialize the supervisor.
@@ -506,39 +521,26 @@ handle_call(which_children, _From, State) ->
handle_call(count_children, _From, #state{children = [#child{restart_type = temporary,
child_type = CT}]} = State)
when ?is_simple(State) ->
- {Active, Count} =
- ?SETS:fold(fun(Pid, {Alive, Tot}) ->
- case is_pid(Pid) andalso is_process_alive(Pid) of
- true ->{Alive+1, Tot +1};
- false ->
- {Alive, Tot + 1}
- end
- end, {0, 0}, dynamics_db(temporary, State#state.dynamics)),
+ Sz = ?SETS:size(dynamics_db(temporary, State#state.dynamics)),
Reply = case CT of
- supervisor -> [{specs, 1}, {active, Active},
- {supervisors, Count}, {workers, 0}];
- worker -> [{specs, 1}, {active, Active},
- {supervisors, 0}, {workers, Count}]
+ supervisor -> [{specs, 1}, {active, Sz},
+ {supervisors, Sz}, {workers, 0}];
+ worker -> [{specs, 1}, {active, Sz},
+ {supervisors, 0}, {workers, Sz}]
end,
{reply, Reply, State};
-handle_call(count_children, _From, #state{children = [#child{restart_type = RType,
+handle_call(count_children, _From, #state{dynamic_restarts = Restarts,
+ children = [#child{restart_type = RType,
child_type = CT}]} = State)
when ?is_simple(State) ->
- {Active, Count} =
- ?DICTS:fold(fun(Pid, _Val, {Alive, Tot}) ->
- case is_pid(Pid) andalso is_process_alive(Pid) of
- true ->
- {Alive+1, Tot +1};
- false ->
- {Alive, Tot + 1}
- end
- end, {0, 0}, dynamics_db(RType, State#state.dynamics)),
+ Sz = ?DICTS:size(dynamics_db(RType, State#state.dynamics)),
+ Active = Sz - Restarts,
Reply = case CT of
supervisor -> [{specs, 1}, {active, Active},
- {supervisors, Count}, {workers, 0}];
+ {supervisors, Sz}, {workers, 0}];
worker -> [{specs, 1}, {active, Active},
- {supervisors, 0}, {workers, Count}]
+ {supervisors, 0}, {workers, Sz}]
end,
{reply, Reply, State};
@@ -808,8 +810,15 @@ restart(Child, State) ->
{shutdown, remove_child(Child, NState)}
end.
-restart(simple_one_for_one, Child, State) ->
+restart(simple_one_for_one, Child, State0) ->
#child{pid = OldPid, mfargs = {M, F, A}} = Child,
+ State = case OldPid of
+ ?restarting(_) ->
+ NRes = State0#state.dynamic_restarts - 1,
+ State0#state{dynamic_restarts = NRes};
+ _ ->
+ State0
+ end,
Dynamics = ?DICTS:erase(OldPid, dynamics_db(Child#child.restart_type,
State#state.dynamics)),
case do_start_child_i(M, F, A) of
@@ -822,8 +831,10 @@ restart(simple_one_for_one, Child, State) ->
NState = State#state{dynamics = DynamicsDb},
{ok, NState};
{error, Error} ->
+ NRestarts = State#state.dynamic_restarts + 1,
DynamicsDb = {dict, ?DICTS:store(restarting(OldPid), A, Dynamics)},
- NState = State#state{dynamics = DynamicsDb},
+ NState = State#state{dynamic_restarts = NRestarts,
+ dynamics = DynamicsDb},
report_error(start_error, Error, Child, State#state.name),
{try_again, NState}
end;
View
52 lib/stdlib/test/supervisor_SUITE.erl
@@ -67,6 +67,7 @@
%% Misc tests
-export([child_unlink/1, tree/1, count_children/1,
+ count_restarting_children/1,
do_not_save_start_parameters_for_temporary_children/1,
do_not_save_child_specs_for_temporary_children/1,
simple_one_for_one_scale_many_temporary_children/1,
@@ -90,7 +91,8 @@ all() ->
{group, normal_termination},
{group, shutdown_termination},
{group, abnormal_termination}, child_unlink, tree,
- count_children, do_not_save_start_parameters_for_temporary_children,
+ count_children, count_restarting_children,
+ do_not_save_start_parameters_for_temporary_children,
do_not_save_child_specs_for_temporary_children,
simple_one_for_one_scale_many_temporary_children, temporary_bystander,
simple_global_supervisor, hanging_restart_loop, hanging_restart_loop_simple,
@@ -1459,6 +1461,54 @@ count_children(Config) when is_list(Config) ->
[1,0,0,0] = get_child_counts(sup_test).
%%-------------------------------------------------------------------------
+%% Test count_children when some children are restarting
+count_restarting_children(Config) when is_list(Config) ->
+ process_flag(trap_exit, true),
+ Child = {child, {supervisor_deadlock, start_child_noreg, []},
+ permanent, brutal_kill, worker, []},
+ %% 2 sek delay on failing restart (see supervisor_deadlock.erl) ->
+ %% MaxR=20, MaxT=10 should ensure a restart loop when starting and
+ %% restarting 3 instances of the child (as below)
+ {ok, SupPid} = start_link({ok, {{simple_one_for_one, 20, 10}, [Child]}}),
+
+ %% Ets table with state read by supervisor_deadlock.erl
+ ets:new(supervisor_deadlock,[set,named_table,public]),
+ ets:insert(supervisor_deadlock,{fail_start,false}),
+
+ [1,0,0,0] = get_child_counts(SupPid),
+ {ok, Ch1_1} = supervisor:start_child(SupPid, []),
+ [1,1,0,1] = get_child_counts(SupPid),
+ {ok, Ch1_2} = supervisor:start_child(SupPid, []),
+ [1,2,0,2] = get_child_counts(SupPid),
+ {ok, Ch1_3} = supervisor:start_child(SupPid, []),
+ [1,3,0,3] = get_child_counts(SupPid),
+
+ supervisor_deadlock:restart_child(Ch1_1),
+ supervisor_deadlock:restart_child(Ch1_2),
+ supervisor_deadlock:restart_child(Ch1_3),
+ test_server:sleep(400),
+ [1,3,0,3] = get_child_counts(SupPid),
+ [Ch2_1, Ch2_2, Ch2_3] = [C || {_,C,_,_} <- supervisor:which_children(SupPid)],
+
+ ets:insert(supervisor_deadlock,{fail_start,true}),
+ supervisor_deadlock:restart_child(Ch2_1),
+ supervisor_deadlock:restart_child(Ch2_2),
+ test_server:sleep(4000), % allow restart to happen before proceeding
+ [1,1,0,3] = get_child_counts(SupPid),
+
+ ets:insert(supervisor_deadlock,{fail_start,false}),
+ test_server:sleep(4000), % allow restart to happen before proceeding
+ [1,3,0,3] = get_child_counts(SupPid),
+
+ ok = supervisor:terminate_child(SupPid, Ch2_3),
+ [1,2,0,2] = get_child_counts(SupPid),
+ [Ch3_1, Ch3_2] = [C || {_,C,_,_} <- supervisor:which_children(SupPid)],
+ ok = supervisor:terminate_child(SupPid, Ch3_1),
+ [1,1,0,1] = get_child_counts(SupPid),
+ ok = supervisor:terminate_child(SupPid, Ch3_2),
+ [1,0,0,0] = get_child_counts(SupPid).
+
+%%-------------------------------------------------------------------------
%% Temporary children shall not be restarted so they should not save
%% start parameters, as it potentially can take up a huge amount of
%% memory for no purpose.
View
14 lib/stdlib/test/supervisor_deadlock.erl
@@ -11,9 +11,11 @@ init([child]) ->
%% terminates immediately
{ok, []};
[{fail_start, true}] ->
- %% Restart frequency is MaxR=8, MaxT=10, so this will
- %% ensure that restart intensity is not reached -> restart
- %% loop
+ %% A restart frequency of MaxR=8, MaxT=10 should ensure
+ %% that restart intensity is not reached -> restart loop.
+ %% (Note that if we use simple_one_for_one, and start
+ %% 'many' child instances, the restart frequency must be
+ %% ajusted accordingly.)
timer:sleep(2000), % NOTE: this could be a gen_server call timeout
{stop, error}
@@ -41,5 +43,11 @@ code_change(_OldVsn, State, _Extra) ->
start_child() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [child], []).
+start_child_noreg() ->
+ gen_server:start_link(?MODULE, [child], []).
+
restart_child() ->
gen_server:cast(supervisor_deadlock, restart).
+
+restart_child(Pid) ->
+ gen_server:cast(Pid, restart).
View
1 lib/tools/src/lcnt.erl
@@ -933,7 +933,6 @@ strings(Strings) -> strings(Strings, []).
strings([], Out) -> Out;
strings([{space, N, S} | Ss], Out) -> strings(Ss, Out ++ term2string(term2string("~~~ws", [N]), [S]));
strings([{left, N, S} | Ss], Out) -> strings(Ss, Out ++ term2string(term2string(" ~~s~~~ws", [N]), [S,""]));
-strings([{format, Format, S} | Ss], Out) -> strings(Ss, Out ++ term2string(Format, [S]));
strings([S|Ss], Out) -> strings(Ss, Out ++ term2string("~ts", [S])).

No commit comments for this range

Something went wrong with that request. Please try again.