race conditions? randomly on custom deployments

[helllamer]

I have an app, that using idna (and ux) very often. I'm generating release using rebar generate. Release usually cannot start with following error (but sometimes it starts - on slower servers):

=ERROR REPORT==== 11-Nov-2012::12:52:51 ===
** Generic server ux_unidata_server terminating 
** Last message in was {get_default,{unidata,to_lower}}
** When Server state == []
** Reason for termination == 
** {{case_clause,<0.454.0>},
    [{ux_unidata_server,handle_call,3,
                        [{file,"src/unidata/ux_unidata_server.erl"},
                         {line,102}]},
     {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},
     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}

=CRASH REPORT==== 11-Nov-2012::12:52:51 ===
  crasher:
    initial call: ux_unidata_server:init/1
    pid: <0.169.0>
    registered_name: ux_unidata_server
    exception exit: {{case_clause,<0.454.0>},
                     [{ux_unidata_server,handle_call,3,
                          [{file,"src/unidata/ux_unidata_server.erl"},
                           {line,102}]},
                      {gen_server,handle_msg,5,
                          [{file,"gen_server.erl"},{line,588}]},
                      {proc_lib,init_p_do_apply,3,
                          [{file,"proc_lib.erl"},{line,227}]}]}
      in function  gen_server:terminate/6 (gen_server.erl, line 747)
    ancestors: [ux_sup,<0.161.0>]
    messages: []
    links: [<0.166.0>]
    dictionary: [{{unidata,to_lower},<0.454.0>},
                  {#Ref<0.0.0.2659>,{comp_exclusions,is_exclusion}},
                  {{comp_exclusions,is_exclusion},
                   #Fun<ux_unidata_filelist.2.116846164>},
                  {#Ref<0.0.0.2603>,{unidata,to_lower}}]
    trap_exit: false
    status: running
    heap_size: 2584
    stack_size: 24
    reductions: 407
  neighbours:

=SUPERVISOR REPORT==== 11-Nov-2012::12:52:51 ===
     Supervisor: {local,ux_sup}
     Context:    child_terminated
     Reason:     {{case_clause,<0.454.0>},
                  [{ux_unidata_server,handle_call,3,
                       [{file,"src/unidata/ux_unidata_server.erl"},
                        {line,102}]},
                   {gen_server,handle_msg,5,
                       [{file,"gen_server.erl"},{line,588}]},
                   {proc_lib,init_p_do_apply,3,
                       [{file,"proc_lib.erl"},{line,227}]}]}
     Offender:   [{pid,<0.169.0>},
                  {name,ux_unidata_server},
                  {mfargs,{ux_unidata_server,start_link,[]}},
                  {restart_type,permanent},
                  {shutdown,10000},
                  {child_type,worker}]

..............

=SUPERVISOR REPORT==== 11-Nov-2012::13:01:08 ===
     Supervisor: {local,ux_sup}
     Context:    shutdown
     Reason:     reached_max_restart_intensity
     Offender:   [{pid,<0.895.0>},
                  {name,ux_unidata_server},
                  {mfargs,{ux_unidata_server,start_link,[]}},
                  {restart_type,permanent},
                  {shutdown,10000},
                  {child_type,worker}]


=INFO REPORT==== 11-Nov-2012::13:01:08 ===
    application: ux
    exited: shutdown
    type: permanent
** Found 0 name clashes in code paths 
{"Kernel pid terminated",application_controller,"{application_terminated,ux,shutdown}"}

Crash dump was written to: ./log/erl_crash.dump
Kernel pid terminated (application_controller) ({application_terminated,ux,shutdown})

[helllamer]

May be, the following error is somehow connected with $subj error.

=ERROR REPORT==== 12-Nov-2012::10:43:52 ===
** Generic server <0.307.0> terminating 
** Last message in was delete_timeout
** When Server state == false
** Reason for termination == 
** no_clients

=CRASH REPORT==== 12-Nov-2012::10:43:52 ===
  crasher:
    initial call: ux_unidata_store:init/1
    pid: <0.307.0>
    registered_name: []
    exception exit: no_clients
      in function  gen_server:terminate/6 (gen_server.erl, line 747)
    ancestors: [ux_unidata_store_sup,ux_sup,<0.151.0>]
    messages: []
    links: [<0.163.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1597
    stack_size: 24
    reductions: 10283
  neighbours:

=SUPERVISOR REPORT==== 12-Nov-2012::10:43:52 ===
     Supervisor: {local,ux_unidata_store_sup}
     Context:    child_terminated
     Reason:     no_clients
     Offender:   [{pid,<0.307.0>},
                  {name,ux_unidata_store},
                  {mfargs,{ux_unidata_store,start_link,undefined}},
                  {restart_type,temporary},
                  {shutdown,2000},
                  {child_type,worker}]

[arcusfelis]

Hi,

I fixed this error:

exception exit: no_clients

And I did not fix the first error. Could you attach the whole SASL log?

First error is about type mismatch, pid from here {{unidata,to_lower},<0.454.0>} is not a function as expected. I will fix it soon.

[arcusfelis]

The error was reproduced in the ux_race_cond_case common test and fixed in 473e862.
Could you check this fix and confirm, that the problem is gone?

[helllamer]

Thanks for your work!

I've stripped SASL log in the middle, because it contains 9 more times
of supervisor restarts with same error. I'll grab full sasl log in
next days (need production server restart).

В Mon, 12 Nov 2012 10:10:52 -0800
Michael Uvarov notifications@github.com пишет:

Hi,

I fixed this error:

exception exit: no_clients

And I did not fix the first error. Could you attach the whole SASL
log?

First error is about type mismatch, pid from here
{{unidata,to_lower},<0.454.0>} is not a function as expected. I
will fix it soon.

---

Reply to this email directly or view it on GitHub:
#4 (comment)

[helllamer]

Thanks!

I'm away from office and cannot initiate server redeployment rigth now.
I will check everything tomorrow (after 8-9 hours till now).

В Mon, 12 Nov 2012 10:30:11 -0800
Michael Uvarov notifications@github.com пишет:

The error was reproduced in the ux_race_cond_case common test and
fixed in 473e862. Could you check
this fix and confirm, that the problem is gone?

---

Reply to this email directly or view it on GitHub:
#4 (comment)

[helllamer]

Ok, all issues looks fixed 100%. Thank you!